Genetic vs Random generation

Hello, I  would like know your opinion and your experiences using the two different methods of generation, it is true that genetic generation will create more overfitted strategy?

Hi,

It is true since genetic evolution is a machine learning system, and those systems “train” on a certain set of data, running on it again and again in order to improve, therefore fitting themselves to the data they are trained on.

Random generation obviously has a much lower risk of overfitting since parameters are generated randomly and then the system is ran on your data set, there is no “training” going on. It will become a problem if you don’t check robustness well enough or if you optimise the strategy parameters too vigorously.

But luckily SQ X has some great features to avoid overfitting:

1) Extensive Robustness testing.

2) Use at least 2 OOS periods which are at least 30% of the data (Which are treated as a “new unseen set of data” for the system, the model isn’t training on it)

3) optimise the strategy only lightly (Only SL or TP coefficient for example)

4) Use a large data set, to make sure you’re not training on “noise”.

 

– I love generating a good initial amount randomly on around 2 years of data (for 1H trading), resulting in about 1000 strategies, and then using it as initial population for genetic evolution with at least 10 islands of 100 strategies each, this time on a 15 years data with a few years OOS. SQ X makes this process a breeze..

Ilya

How do you set 2 OOS periods, it is automatic or you retest the strategies adding the second OOS period?

In SQ X you can define as many OOS as you want on data tab, if you want to integrate it into generation. Otherwise you can just leave out the latest 6 months during building, and then retest the models on those 6 months in the retester, for example. It’s the same thing.. just a matter of comfort and filtering preference (Ranking). So for example you can define 2 filter systems during building, one for IS & one for OOS, therefore automating the first retest process.. for example:

 

Ah yes, I just found the option to set more than one OOS, thank you.

I use in sq 3.8.2 only genetic generation.

The reason is clear. With genetic generation generation you will find better strategies.

 

You can make some tests.

a) generate random  b) generate genetic with the same building blocks

Compare the results after robustnes tests

 

explanation:

The seeking space for finding good strategies is too high for random generation. You have to wait millions of years if you find good strategies.

Genetic is more targeted. If you have a good search function for the genetic algorithm. The target function contains for example (profitfaktor, stability, Retdd, maxDD)…

 

===============

4.115

In SQ 4.115 I am not sure if random or genetic is better. I think in SQ 4.115 are too many bugs at the moment.

Possible that in the genetic algorithm are errors too? I don´t know?

====

In general I prefer genetic generation

thomas

its not a question of what is better or worse – its totally different approach

every genetic building will start with random generation of starting population and trying to find better strategy with some methods – you need to know that with this method you will get much quicker to results but the problem is overfitting – its as Ilya said

but be aware – you dont need better backtest, you need more robust strategy, which has probability to work in the future – past behavior doesnt matter. “best” strategies will be mostly overfitted and fail

Supposing to use genetic evolution, SQ4 has enough instruments to find the strategies really not overfitted??

I have a little of experience using an other similar software, and after several months running few strategies in a small account, I dismiss them because the result were not the same of that one showed in the backtest.

Of course before to run those strategies I tested them with some manual robustness test but maybe this has been not enough, and this is the reason why I wanted start this experience with SQ4, but of course I’m full of doubts..

yes overfitting is the big problem. 99.9% of all generated strategies are overfitted.

Genetic Building ist the overfitting machine.

But in random generation is the same problem with the overfitting. Not so big but is available

So, could it be a good idea start with random generation and, only in case no one good strategy has been found, continue to evolve them through genetic evolution?

I feel very different about random gen Thomas, especially with SQ X which is so much faster. When you generate a population randomly, say 200 strategies and then are trying to improve it through 20 generations for example, if it fails to reach the desired fitness and your filters (which mostly will happen), you are basically wasting time on backtesting 4000 strategies which are thrown away, compared to 200 when using random gen, therefore wasting X 20 more time. With random gen, on a 32 thread system, I need 1 day to generate 1000 viable strategies (PF > 1.5, Ret/DD > 2, Stability > 0.7, Winning%>50%, and the below mentioned condition) For me, genetic evo is fabulous way to generate even better strategies on larger data, using a very good population which was generated randomly, but using genetic from scratch, so far to my experience, was mostly a waste of time.

Quick tip (Credit to Notch): Random generation with a small robustness test IN BUILDING PROCESS (randomise trades order randomly with 99% confidence level > 1), will yield much higher quality strategies in random gen..

Ilya

I have another question for you that for me is not clear: following the course from Zdenek Zanka he said that strategies generated without use of take profit have better performance in backtest but usually less in real trading, while strategies with take profit have more possibility to reflect the performance showed in the backtest. Is true for you this affermation, I’m not able to understand the logic behind it.

Thank you so much for sharing this tip. As a new user of SQX I find this very creative and extremely helpful! I will let you know how it goes.