100% automated and 100% accurate SQ workflow test case

My first 100% automated and 100% accurate workflow, StrategyQuant ‘custom projects’ test case

DISCLAIMER: The presented results below are still preliminary, there is still a small chance that my positive results are influenced by an undiscovered bug in the current version of SQ-X (build 118.84) or that I’ve just made a stupid mistake somewhere in my workflow resulting in a huge ‘Data Mining Bias’. However I did my best and rechecked everything multiple times…Moreover since this all is based on a relatively new ‘custom projects’ feature of SQ-X, nothing of this has been tested yet on a real account…but I think I have built a strong case supporting I could be right on this one

My claim: It looks like I’ve managed to create a 100% automated and 100% accurate workflow using StrategyQuant feature called ‘custom projects’.

100% automated means: I push 1 button before going to bed, and every morning my workflow automatically generates, validates and selects few new strategies which are ‘ready to go’

‘Ready to go’ means: I can deploy them immediately to my live account. Without a need of further processing.

100% accurate means: Every single strategy that has been selected by this automatic workflow (~50 so far), has been profitable in the 2 years period from the generation date.

To test my workflow I’ve adapted my SFT method as described in this topic: See HERE.

The workflow is based on standard validation test (common knowledge) as shared by SQ team in their free courses, however with a very rigorous settings. The workflow does not use any advanced validation methods like WFA,WFM,OP,SPP. Instead a customized Monte-Carlo test is used to simulate behavior of a SPR method. No portfolio analysis is performed (some systems can be correlated!).

My automatic workflow test case is split into two verification periods:
1. End of year 2014.
2. End of year 2016.

At each point in time 1 and 2, I used my workflow to automatically generate and automatically select 20 NEW strategies (out of several hundreds thousands systems) without ANY manual intervention and then ALL of selected systems where forward tested using SFT (future data). Let me be clear on one thing: I did not cherry picked any strategies.

It seams that every single selected strategy was profitable in the period following the selected generation date. See figures blow:

Test case 1: Strategy creation @ 2014.12.31, Simulated Forward test: 2015.01.01…2016.12.31.
Real-Ticks (Dukascopy data), Real-spread (no commissions)

Test case 2: Strategy creation @ 2016.12.31, Simulated Forward test: 2017.01.01…2018.12.31.
Real-Ticks (Dukascopy data), Real-spread (no commissions)

My conclusions so far:
1. If there are no mistakes, then it seems that it is totally possible to use SQ-X automated ‘custom projects’ to automatically generate and select profitable trading systems.
2. No advanced validation/filtering methods needed. Of course these tests should only improve total result and minimize DD on portfolio level.
3. The results in SFT of >2014 are slightly better than >2016. Workflow is somehow sensitive to used data during strategy generation (due to changing market condition). It seems that years 2017 and 2018 are very difficult years for trading using the selected trading type.
4. It is not 100% proven yet, but it’s a pretty damn good result so far, taking into account it’s based on a simple workflow that is using basic filtering principles.
5. Some of the strategies can be correlated, but for the sake of this investigation no manual correlation filtering has been performed. This would jeopardize the objectivity of this test case.
6. The filtering settings are very rigorous, this workflow filters out only the most robust strategies. According to my statistics only 0.05% of the generated strategies are able to pass this workflow.

TODO:
– Refine the workflow and implement further strategy selection, perform correlation tests, WFM analysis and additional portfolio level related tests.

– I’m also waiting for 01.03.2019 and build 119 with new features 😉

Greets,
Chris

Yup if they are all winners in one OOS period they can likely all be loosers in another.

hankeys wrote:

after 2 years of live trading – 1M and TICK backtest is still holding, but real EQ curve is already in loss…NFPs, gaps…and they got ya

Or maybe because of this bug if I’m right! This bug will cause an EA back-test to behave differently from the same EA on a live account. If you happen to have more than one EA on one account, they interfere with each other. Many of the sqx ea functions seem to be reliant on global variables but when you have more than one EA on an account then they (occasionally – once randomly every 20+ hours per EA) delete each others GV. GV seem to be used for important functions such as trailing stop and timed exits! How frequently such interference happens depends on how many EA you have on one account…

https://roadmap.strategyquant.com/tasks/sq4_5095

False alarm actually ignore my previous post. I was wrong.

@Coensio

First of all, I want to thank you for sharing your thoughts with us. But I don’t understand why you’re crazy now.
When you publish your work, you must be able to defend it. That there are some unqualified comments is unfortunately normal.

But now to my question:

When I look at your screenshots I notice that the drawdowns of the different years are partly completely identical. Are you sure that it’s resilient?
Also the NetProfits are extremely different! Why?

Marcel wrote:

First of all, I want to thank you for sharing your thoughts with us. But I don’t understand why you’re crazy now.

No really, I’m not crazy, but usually I just avoid arguing with other people, because in most cases I just do not care 😉

So yes, everything what you see is true, I did not draw my results in mspaint…really, but there is a ‘catch’ (or even two) to this method:

1. This workflow targets only one specific trading type: breakout/(short term)swing trading, so it also targets only specific indicators and settings. What you see is a result of genetic generation and heavy robustness testing (and high Ret/DD filtering), this means that the resulting strategies are somewhat correlated. Basically it is one type of trading method using different indicators/settings. So that is why you see some similarities, but also different profits.

2. Those results are only true when the ‘OOS’ period following the workflow years has similar market conditions and sufficient volatility. It worked perfectly in 2014-2016 even 2017, but the results are not that pretty in >2018. In those years I see nothing but stagnation…volatility is gone…

But this is normal, markets will always change…we have now ‘quiet years’..in the mean time I’m exploring different markets where breakout trading still works pretty well…

That may be true, but it’s highly implausible that the same strategy has exactly the same drawdown in the following year, or do you see it differently?

Furthermore, I don’t understand how it can be that you turn on SQX in the evening and have stable strategies in the morning.
Let me explain this briefly:

Last month (start 16.06.2019 to 13.07.2019) I searched almost 100000000 (1 trillion) strategies through SQX, looking for a strategy that has the following qualities:

Market: Eur/Usd / 1 hour
Period 01.01.2019 – 15.06.2019
Profit factor: => 1.3
RET/DD => 0.5
MAXDD<= 25%
Stagnation<= 25%.
Stability <= 80

The result: 0 (!)

There was no strategy at all that met these criteria.
As mentioned above, 1 trillion strategies were tested!

For this reason I am very surprised how you can say that you turn on SQX in the evening and that you can have more than 30 stable strategies in the morning.

First of all:

You would get yourself a very strong “popularity boost” if you didn’t always judge everything without being asked.

Secondly:

Thank you very much! To be honest, I’m confused as to why I don’t get a single one displayed as suitable for 1000000000 strategies.

Would you please send me the settings you were looking for?

@ Marcel, looks like 1 Billion , no matter thought.  I did the same but on longer data 1986-2018-09.  Testing those same period i end up with  more then 3000 passing your criteria which is less then 10% but I traded 37 of them between 0612-0712 and they made 21.5% on a real small account. But that was a good month actually 90% of 35 k strategies made a profit. Month before  less then 10% was profitable.

But the question is, why out of a billion strategies not a single one ended up in the database…..that confuses me a little….

@Marcel:

SQX is not different than any other thing based on computers….where there is this law that says:

“garbage in” = “garbage out”.

Basically you need to know what kind of indicators, with what kind of settings to use to target specific trading method. When done properly, you can generate a large amount of interesting strategies in a few hours. Seeing my results, now I realize these are very mediocre results, nothing special. So I think you must be doing something wrong, very wrong…

To be constructive maybe you should share your settings, and maybe someone will look at it and tell you what is going wrong.

attached

Those are the stats of rejected strategies….after few minutes you can already see where your issues are located….so basically you try to build strategies using filters, what in your opinion should reflect a ‘perfect’ strategy using 14 years of data and 3 pips spread…hmm this will probably not happen..instead try to find good strategies on a much shorter period of time and run WFA on them…

I think you’re wrong because it wasn’t 14 years, it was only 1.5 years.
Possibly SQX does not change the previously set period with the settings 🙂

So I search already in extremely short periods of time (from 01.01.2019 – middle of June) and 1 trillion strategies were rejected…..Why?

MAXDD<= 25%
Stagnation<= 25%.
Stability <= 80

filtering by % doesnt make sense, because its base dependent

stability 80 – means what? stability is in decimal points, and 0.8 will mean almost straight line

conditions 1-5…this is very complicated to have strats with 2 and more conditions…for me 2 is max for conditions…also i dont like exit conditions at all

max period of indicator only 30??? for M15…

i dont like your genetics settings…you know why you set it by this way and what everything means?

why do you use only a few building blocks?

so i see many questions here…

and i dont see the basic point generating only on a few dataset – for me minimum for IS+OOS are 2+2 years, better 3+3