“Tell me, and I will forget. Show me, and I may remember. Involve me, and I will understand.”

When I and my cat set off to developing a financial exchange engine, this very quote of Confucius was ringing in my head. We were a dream team: motivated, solid, result-oriented. Our condition could be best described as interest and readiness to take on challenges on our way to MVP. All those challenges and discoveries, met during development, have adjusted the whole process and were well worth the time spent. Today I’d like to share some of them with you — and tell about the results of this experiment as well.

For the impatient, here is the link to the self-hosted demo. The rest, please, welcome to read further.

The main challenge of this project was the limited resources. If this limit hadn’t been that critical, the following list would, probably, look a bit different. My insights might seem banal, but each question can be treated differently. There is a huge divide between just ‘hearing about a problem’, ‘becoming aware of the problem’ and ‘understanding the problem’.

So, here is the list:

It’s way harder to bring a project to MVP than just to start it

When you are working on a project single-handedly and, what’s more, in your free time only, it’s very easy to shift the deadlines. I personally had to stop for half a year because of the main workload and family issues.

The project implementation was divided into two parts: March-April 2019 and November-December 2019. In November I wasn’t sure that I would come to MVP. However, all the internal interfaces were ready, as well as the storage system and order processing. All I had to do was create user interfaces. So, I decided to move on.

As a result, I realized that it’s better not to put projects on hold as the initial motivation is lost and the focus can be shifted. To keep going, I had to put a real effort in finding time for the project again.

Marketing can be different

This insight appeared after I’d been researching and choosing the frontend stack. Now there is a wide variety of libraries and frameworks whose APIs are changing so rapidly that it might make you stunned.

The hype around any of them is not an indication of quality. The size of the community around a project often doesn’t correlate to the quality of the solution. If a project/solution/technology position themselves as the best of the best and are described as ‘magical’ and ‘amazing’, it’s better to take it with a grain of salt or even avoid. It is likely to save you some time in future.

You never know which problem of failure in third-party software you might encounter

Even proven technologies can let you down. It happens so that an instrument you’ve been using for a long time starts acting not the way you want. To my surprise, it happened to me while I was working on this project too. There was Timescale and its time_bucket which didn’t support big intervals; Tarantool and replication issues and manual control of transactions; Rustler which started copying more data while calling functions...this list can be continued. What I realized here was that in the open source world you can always manage to fix the problem or find a way to avoid it by digging deeper into the source code of the app.

When resources are limited, you can’t but plan the amount of work carefully. If a problem turns up, you have do deal with it on your own, be it a small one or big one. You just have noone to delegate it to.

So, every technical difficulty — for instance, a mistake in a third-party library, or a limitation in a technology you are using, or sometimes an issue of architecture — can seriously shift the deadline.

Your eyes might glaze over and affect decision making

Usually, in case of commercial development, we communicate with other people. Any project involves people who are making it or using it. During development, we constantly get updates from other members of the process which add up to our perspective. Making any technically challenging decision is a complex process. When you are absorbed in a problem and spend hours or even days trying to find the ways of addressing it, you sometimes can’t keep your focus.

I’ve realized how important and valuable working in a team is. An opportunity to discuss a question with a colleague who isn’t working on the same task but is still in the know can be really useful. They will be able to assess your approach or solution critically, and it will save your time and reduce the likelihood of mistakes.

Teaching programming to a cat is a tricky business

Project outcomes

Before I started this experiment, I’d set 2 main goals:

• A deeper understanding of the subject and improvement of technical expertise;
• Identifying strengths and weaknesses of functional languages and open source projects in trading systems development.

During the experiment I definitely managed to deepen and systematize my knowledge and expertise and even tried to share the outcomes with the community — by the way, you will find the links to the previous boringly theoretical articles in my profile. So, this goal has been achieved successfully.

As to the functional programming...well, I’m looking from Erlang/OTP perspective now. Erlang as a language and OTP as a framework are suited to industrial programming and financial systems implementation. First, the ecosystem is quite mature. Second, minimum syntactic basis of the language, immutability, and approaches embodied in OTP enable for quick implementation of scalable, distributed and reliable systems.
Unfortunately, I didn’t keep a time log, but the whole process took me about 4 months while working after having done my day job. Meanwhile, there are more than 200 integration tests in the project.

Sounds good and all, you will say. However, there still is a small fly in the ointment. Erlang is pretty similar to Python in terms of performance. Unlike Python, it lets us write highly concurrent systems with an adjustable level of parallelism. Also, such systems are easy to scale, and the components can be distributed within a cluster. The systems turn out to be of high capacity, but not superfast.
Basing on the last conclusion, I realized that the most effective and productive approach would be not distributing market handlers and client accounts to different machines within a cluster, but processing all markets and accounts within a single machine with data replication for the sake of reliability.

Due to the limited performance of Erlang, market processing should be done with the help of lower-level languages without GC and fair schedulers. In current implementation I have already moved some part of the functional to NIF on Rust.
This way, thanks to the optimized work with network and binary structures, Erlang perfectly solves the issues of organizing a reliable and fast platform. An issue of working with orders and accounts, though, is better to be solved with the help of Rust/C/C++ (to your taste).

Last but not least is the choice of the language for frontend. Old habits die hard — I opted for Vue + Js. However, if I were choosing the stack now, I would prefer Vue + Typescript. Stricter typing would make it possible to accelerate the development.
All in all, thanks to everyone who was following the experiment. I hope it wasn’t too boring, or was it? Stay tuned — it’s not goodbye! :-)