You never avoid some kind of curve fitting in my opinion. That’s all we have as traders. We build models and we verify them on history. Even if you later use some extra data to verify that strategy (data never used in SQ) you also fit the strategy to that data. If the strategy does not work on that data you probably won’t use it but you will if the strategy ‘fits’ to that data :)
this is a bad idea
manual inefficient work
just stop SQ to peek into the future OOS and be honest with backtests and if OOS is no good then drop it and move on
- This reply was modified 11 months, 2 weeks ago by gin.
I’ve always asked to myself the same question: If you select an OOS range data, will SQ actively blend strategies on the IS results to get a better fit on OOS? Or will just discard those that get the best fit on IS and don’t achieve the minimum criteria selected on OOS?
I suspect that SQ works on the first way. So if you choose OOS, the Building process is influenced by this OOS because it forces the strategies to the best OOS instead of building on base only to the IS period. That’s my guess. So, if I’m right it would be better to build only with IS and make a second run on the data range that would be your OOS
using only IS without some robustnests tests and using genetics is a first way how to start loosing
i tested many and many approaches and yes, i can tolerate opinions that you dont need some OOS testing, but without this check on never seen data of your model (workflow) you have no idea how your model is robust to the future
You want to be a profitable algotrader? We started using StrateQuant software in early 2016. For now we have a very big knowhow for building EAs for every possible types of markets. We share this knowhow, apps, tools and also all final strategies with real traders. If you want to join us, fill in the FORM. 1500+ final SQX strategies for members running on demo accounts to verify the edge DEMO ACCS. We provide also strategies for indices - DAX, DOW JONES, NASDAQ, SP, UK, because we have realtick data from our brokers.
same problem for me… when i use OOS in the builder, genetic algorithm seems to look at the IS+OOS fitness to generate new strategies…
For exemple, if i use OOS in the builder with criterias like ret/DD(IS) > 2 and no criteria for OOS and let it generate 100 strategies, they all have good OOS.
But if i use only IS with the same criteria and after, retest my 100 strategies on OOS, 99% are overfit and crash in the OOS area.
So, for me, OOS is absolutly useless in the builder :/
using only IS without some robustnests tests and using genetics is a first way how to start loosing i tested many and many approaches and yes, i can tolerate opinions that you dont need some OOS testing, but without this check on never seen data of your model (workflow) you have no idea how your model is robust to the future
why don’t you like genetics?
you don’t believe in evolution and natural selection?
on the other hand there is a big problem.
If you generate strategies with the “backtest on additional market” activated, the builder erase the IS fitness and OOS fitness for look only at the IS+OOS fitness.
So, if we use additional markets in the builder, the OOS become fake OOS and the builder overfit the strategies.
If, like me, you use precedent generation as initial population, that break all the process and cancel your OOS area.
I think you are wrong. After reading you I was concerned because I use several Crosschecks on my Buidling process and by any means I wouldn’t want that the optimization process would include the Additional Markets. So I took the SQ manual. According to the manual, ” the strategy is evolved only on the In Sample part of data”. So the Genetic Evolution; that is the optimization process. It would be working just on IS – at least I hope so!
I’ve built and am testing quite a number of portfolios for M5 H1 and H4 in MT4.
The strategies in the portfolios range from heavily robustness tested strategies with WFM, MC etc. to no OOS at all.
By far my best performing portfolio is my H4 portfolio consisting of 96 strategies accross 12 FX pairs and gold (13 pairs all up).
This, ironically, is the portfolio I built consisting of strategies built using only the builder task using genetic evolution with no OOS. None of these strategies are robustness tested either. All I did was build them using ridiculously high spreads and the maximum data available.
I wanted to run this experiment because I figured if I can get a really good equity curve for each strategy, with spreads I’ll most likely never see in real trading, over a very long period of time, on a high time frame where there is likely to be far less noise, that even though this is “curve fitted”, it will need to continue to have performed in a large array of varying market conditions for a pair.
Seems to be working very well :)
- This reply was modified 10 months, 2 weeks ago by Insanity82007.
You must be logged in to reply to this topic.