Optimizer in OsEngine. A few words about robustness.

With the ability to quickly and easily optimize strategies with thousands of different parameters, some strategies will naturally be "over-optimized" and will not reflect any market inefficiencies. On the contrary, most strategies that have been optimized will become non-functional.

In this context, we will have to talk about Robustness.

1. Robustness is.

The term "robustness" refers to the ability of a trading strategy to replicate the results of its backtesting on other data.

Example 1.

You tested a strategy in the tester and see the result in a red square. Great! You implemented the strategy in trading, and in real time over the next two months, the strategy gave you approximately the same profitability as in the tester:

Fig. 1. Strategy with high robustness. Replicates test results in real trading.

Example 2.

You tested a strategy in the tester and see the result in a red square. You implemented the strategy in trading, and in real time over the next two months (green square), the strategy resulted in losses:

Fig. 2. Strategy with low robustness. Does not replicate test results in real trading.

Issues.

This term arose at the moment when there was a technical possibility to test and optimize trading strategies. It quickly became clear that not all strategies that show good profit in the tester show profit in real trading. Most strategies that show profit in the tester do not show profit in reality.

Example of Alex Wang.

In 2012, Alex Wang wrote a program for automatic pattern search. It was called "StockPatternViewer." You can download it from the internet right now. 

The essence of it was that the program automatically selected a candlestick pattern on the price chart and checked its profitability in the past. 

And so, by clicking a few buttons, dozens of different formations were found that gave huge profits. From 1% profit per trade.

Inspired by these discoveries, Alex Wang launched the found formations in trading and LOST 30% of the deposit.

That's how he got acquainted with robustness... Through pain and money losses.

2. Increasing robustness. Rule 1000 trades.

Later, Alex Wang became concerned about the problem of additional testing of found profitable patterns and divided the tested history into two parts. 

The first - where he searched for profitable patterns.

The second - where he checked whether the pattern would be profitable on unaccounted data.

In fact, this is how he discovered Walk-Forwards tests, which will be a whole chapter here.

It quickly became clear that profitability on a sample not from the training directly depends on the number of trades the robot made in the training sample.

Fig. 3. If you have more than 1000 trades, it is likely to work.

Fig. 4. If you have less than 1000 trades, it is likely NOT to work.

Since then, this has been the main measure of the robustness of a trading strategy. Alex Wang paid for this with his deposit. 

Therefore, the MAIN RULE OF ROBUSTNESS: THERE MUST BE MORE THAN 1000 TRADES!

It doesn't matter how you got these 1000 trades, on one instrument or on several. This is also important, but we will talk about it in the next chapter.

It is also important to understand that this figure is not an axiom. 

The more trades, the higher the robustness. And the fewer trades, the lower the chance that it will make you money in the future.

3. Increasing robustness. Cross-testing.

One of the ways to reach the coveted 1000 trades on the training sample is cross-tests.

This is when you run the same strategy with the same parameters, with the same timeframe, BUT ON DIFFERENT INSTRUMENTS!

Fig. 5. Cross-tests. Tests of the same strategy with the same parameters on different securities.

Such an approach to defining the robustness of strategies was predominant until 2021. It works and allows to identify truly robust trading strategies.

The only drawback of this approach is the enormous complexity. But at some point, a more advanced form of robustness search emerged - Walk-Forwards.

Meanwhile!

When testing Arbitrations and Screeners, we still actively use this technique, and it proves to be successful. So do not forget that such an approach exists, and use it.

4. Increasing Robustness. Walk-Forwards.

A special mechanism of the optimizer's operation that allows to understand how robust the strategy is in an automatic mode.

Our next article will cover this...

If something doesn't work out or if you have any questions, please write in the support chat, link.