Cross checks – robustness tests and analysis

Author: admin

March 1st, 2019

1 Star2 Stars3 Stars4 Stars5 Stars (1 votes, average: 5.00 out of 5)

Cross checks – robustness tests and analysis

Overfitting or curve-fitting strategy to the historical data on which it was build is the biggest danger of strategies generated using any machine learning process.

During or after developing new strategy you should make sure your strategy is robust – which should increase the probability that it will work also in the future.

What is robustness?

It is simply the property of strategy of being able to cope with changing conditions:

  • First of all, strategy should work on unknown data (if the market characteristics didn’t change) either with or without periodic parameter re-optimization
  • It should not break apart if some trades are missed.
  • Robust strategy should not be too sensitive to input parameters – it should work even if you slightly change the input parameter values, such as indicator period or some constant or if historical data are slightly changed – spread or slippage is increased, and so on.

How to test strategies for robustness?

Make use of In Sample and Out of Sample periods

the most basic test for robustness is testing the strategy on Out of Sample data. If you run genetic evolution, the strategy is evolved only on the In Sample part of data. The Out of Sample part is “unknown” to the strategy, so it can be used to determine if the strategy performs also on unknown part of data.


The blue part of each chart is the Out of Sample (unknown) data., We can see that the strategy on the left performs well also on this part, while strategy on the right fails on the unknown data – it is almost certain to be curve fitted.

Leave a Reply

Please Login to comment