The entire community of trading robots developers is brewing around Python and C#. When asked why they chose these languages, they begin to indistinctly grumble about multithreading, the number of ready-made libraries and even about semaphores. So I decided to try to get into this hodgepodge. Yes, it is a hodgepodge, otherwise it cannot be called.

In general, there are a lot of bulky tools that seem to be suitable for work, but somehow they have everything crammed in different corners and exist in a creative mess. At the same time, no one asks the question «Нow and why does it work like that?». Some are just trying to write something in order to make some money, if they are lucky.

As is common with many developers, I was wondering what JavaScript and V8 with JIT are capable of. Can it give you the speed you need for complex math? Initially it started more as a research project. And the case, by the way, was a year and a half ago.

So, what we need to develop and launch a trading strategy, for example, at Binance:

JavaScript technical indicators. It's good that they are, but not in abundance. Let's take the most popular technicalindicators by downloads;

Something to work with Binance, their binance-api-node library.

Take the dumbest strategy in the world. For example, 2 SMA - fast and slow, which tend to converge again in case of divergences.

Despite the stupidity of the strategy, it is necessary to optimize cleverly: either Monte Carlo or with the help of genetics. Let take genetics, 'cause it just sounds nice.

Few details about the strategy and the divergence of the two SMA lines. It’s based on the market's desire for corrections. If the SMA with a faster period goes below the SMA with a slow one, then the market has made a sharp change in price, which will correct backward with a certain probability. In the picture below, these discrepancies are indicated by arrows. Almost all of them have a reverse movement. This is a good entry point to buy stocks. Short positions can be entered when the fast SMA has moved sharply above the slow one.

In general, there is a plan. You just need to write some kind of transport module for higher-level work and a genetics module to adapt the algorithm for financial tasks.

After 3 months there was a sound: «I was born», and the strategy began to slowly trade on its own. At this time, some data on its work was collected, the handles were twisted, supports and limiters were installed. For example, it became clear that it was impossible to trade on the premarket or that it was not possible to survive the 30-minute blocking of the exchange with a sharp jump in prices. In general, nuances were literally everywhere. It was necessary to devote time to the optimizer, which has helped to turn the knob in the strategy (more on this later). Let's start with a quick look at genetics.

The genetic algorithm allows you to simulate the natural process of evolution of a population of living beings. For example, we'll have a population of butterflies. This population multiplies, survives, shares genes with each other, reads Reddit and so on day by day...

The library of genetic algorithms is about the same. To avoid getting bored, here is a series of pictures as «butterflies» are trained to fly to the desired point.

A lone butterfly with random parameters performs pirouettes of Brownian motion, moving in all directions.

Applying a bit of genetics, you take 100 butterflies and cross 20 generations, give them candy for the approach to the point. If they move in the wrong direction, you text is blocked for cruelty to butterflies. We obtain the next slide.

I hope it's clear how this works with butterflies. Now let's talk about trading.

Instead of flying insects we will have a trading strategy. The mathematical expectation of a win is usually taken as the evaluation criterion. This is a characteristic in the theory of gambling, it predicts the amount of winnings that a player can earn or lose for each bet on average. In the gambling language, this is sometimes called «player advantage» (if it is positive for the player) or «house edge» (if it is negative for the player). This is what we need to understand whether we are winning in the exchange with our trading strategy or not. So this will be the main criterion for genetic optimization in our system.

Here is the formula for calculating the expected value, just in case:

Math Expectation = Probability of profit x Average profit - Probability of loose x Average loose

I will allow myself a few code inserts. You need an interface to describe the strategy parameters to twist them. It is presented below as an object literal with different fields.

export const parameters = {

stopLoss: { min: 0.2, max: 9 },

takeProfit: { min: 0.2, max: 9 },

openPercent: { min: 1, max: 15 },

fastSMAPeriod: { min: 2, max: 30, int: true },

slowSMAPeriod: { min: 10, max: 30, int: true },

};

The interface allows you to specify the values that a particular parameter takes, for example, integers or not, even or not, booleans or numbers. Now genetics will know how to fill our population of 100 or 500 individuals with randomly generated parameters. By the way, about the population, in our case one trading strategy with a certain set of settings (genes) will be as an individual that we will save and transfer in crosses from parents to children.

We need to generate 100 random configurations, create trading strategies, pass them the desired configurations and run back testing when we create a population of 100 individuals. Obviously, it took to write the history of unloading for it and everything necessary. Each pass of the strategy further is estimated according to the expectation formula, crossing of individuals is performed (exchange of genes aka parameters) and the next generation is obtained. And so as many times as you like, for example 50. In the end, the results obtained will trade better and better by adjusting the parameters of the fastSMAPeriod and slowSMAPeriod periods, as well as stops and takes.

I write this article in a light and lighthearted manner, avoiding possibly unnecessary technical details. In fact, a whole year of hardcore development has already passed and the experiment got out of control and drove a group of enthusiasts crazy.

During the year it became clear that a library of genetic optimization is not suitable. Their approach works, but it is not complete: there are not enough selection methods. It is how pairs of individuals will be formed for breeding children. For example, whether to use random samples or pair only the strongest and so on.

Also, there is no protection against duplicates, when, due to errors in technical emulation, identical individuals from different families appear as like as two peas. It's like meeting your non-relative doppelganger at the mall. It is clear that in nature this is possible, but there are not 5 parameters that are involved in the formation of an individual, but a billion  ;) The library was not able to do many other things described in this article. In addition to the necessary algorithm settings, work with asynchronous assessments of an individual was not supported. That is, it was impossible to just take and request a story, drive out a strategy, and then say: «Well, I'm ready to be evaluated». I had to make my own library in the absence of those. Along the way, it was necessary to be worn out with performance, because it was required to squeeze all the juices out of JavaScript.

The result is an async-genetic lib that meets all standards, and even began to work faster, not only in terms of code, but also in terms of problem solving. Due to additional settings, for example, it solves the "guess what word I have guessed" problem 2 times faster (on average).

There have been no pictures for a long time, so it's time to show the result of visualizing the strategy and opening deals.

It took a lot of time to write a small trade visualization system for you to see this picture with trades and two SMAs and the loaded history. And then it became clear that the experiment had already gotten out of control and began to live its own life. Of course, by this time, something was already working on the exchange and was trying to make some money. I must say any earnings motivated like nothing else to continue this difficult business.

Initially, genetics worked well, but for a very long time, which seemed strange to me and I began to deal with the problem. After debugging the NodeJS application, I found out that the main problem is the indicators, which are written badly. Should I write my own again? At this stage, the project has already begun to form an ecosystem around itself and the original research direction began to collapse. After that, the focus was on making money and creating a full-fledged platform.

At the moment, I already had my own genetics library and the add-on based on it for optimizing trading strategies. But now indicators were also required. Fortunately, they are not too complicated, not more complicated than what has already been done, anyway. And now, after some time, everything is ready! There are indicators and there are tests for them, because it is very important that they work correctly, as this is the heart of all strategies. Another performance pass showed that now the slowest place is new Date (). Obviously, all this was immediately rewritten for numerical data storage. In general, measurements of the speed of the indicators showed the following:

technicalindicators CCI x 918 ops/sec ±5.81% (80 runs sampled) 
@debut/indicators CCI x 31,471 ops/sec ±0.94% (89 runs sampled)

technicalindicators SMA x 4,683 ops/sec ±1.97% (77 runs sampled) 
@debut/indicators SMA x 63,978 ops/sec ±1.06% (87 runs sampled)

technicalindicators EMA x 29,368 ops/sec ±1.25% (86 runs sampled) 
@debut/indicators EMA x 93,024 ops/sec ±0.99% (90 runs sampled)

 
There are only 3 indicators, hands did not reach to check the rest, but it was already clear that everything was working faster. The main advantage is the specificity of the indicators. In this case, they were written not for graphs, but for streaming calculations, and the results of previous calculations are used to the maximum, counting on the movement always from left to right in time. This is how our own indicators appeared.

After this moment, it became clear that a finished product was obtained for developing strategies. It only remained to refactor it all 15 times to prepare it for the open source.

Before I start advertising, I would like to tell you more about the strategy. In general, we did not abandon the strategy. It still works somewhere to this day, but, of course, we managed to make money mainly on much more complex strategies.

As a result, we created a whole system of tools and called it Debut.

Debut is an ecosystem for developing and launching trading strategies. This is an analogue of the well-known ZenBot, but with much more flexible possibilities for constructing strategies. All you need to do is come up with and describe the entry points to the market and connect the necessary plugins for work. Everything else is a matter of technology: genetic algorithms will help you select the most effective parameters for the strategy (period, stops, and others), the ticker selection module will help you find an asset (token or stock) suitable for the strategy, on which it will work best.

Debut is based on the architecture of the core and add-on plugins that allow you to flexibly customize any solution. The main goal of the entire Debut ecosystem is to simplify the process of creating and launching working trading robots on various exchanges. Debut can working with Binance out of the box.

The project has two starting trading strategies «For example» how to work with the system. Here is a repository with samples and examples.

A year and a half of development led me to many discoveries, some of which remained behind the screen, But I will definitely tell you about them in some report at the conference.

Enjoy discuss in Reddit