100% automated and 100% accurate SQ workflow test case

My first 100% automated and 100% accurate workflow, StrategyQuant ‘custom projects’ test case

DISCLAIMER: The presented results below are still preliminary, there is still a small chance that my positive results are influenced by an undiscovered bug in the current version of SQ-X (build 118.84) or that I’ve just made a stupid mistake somewhere in my workflow resulting in a huge ‘Data Mining Bias’. However I did my best and rechecked everything multiple times…Moreover since this all is based on a relatively new ‘custom projects’ feature of SQ-X, nothing of this has been tested yet on a real account…but I think I have built a strong case supporting I could be right on this one

My claim: It looks like I’ve managed to create a 100% automated and 100% accurate workflow using StrategyQuant feature called ‘custom projects’.

100% automated means: I push 1 button before going to bed, and every morning my workflow automatically generates, validates and selects few new strategies which are ‘ready to go’

‘Ready to go’ means: I can deploy them immediately to my live account. Without a need of further processing.

100% accurate means: Every single strategy that has been selected by this automatic workflow (~50 so far), has been profitable in the 2 years period from the generation date.

To test my workflow I’ve adapted my SFT method as described in this topic: See HERE.

The workflow is based on standard validation test (common knowledge) as shared by SQ team in their free courses, however with a very rigorous settings. The workflow does not use any advanced validation methods like WFA,WFM,OP,SPP. Instead a customized Monte-Carlo test is used to simulate behavior of a SPR method. No portfolio analysis is performed (some systems can be correlated!).

My automatic workflow test case is split into two verification periods:
1. End of year 2014.
2. End of year 2016.

At each point in time 1 and 2, I used my workflow to automatically generate and automatically select 20 NEW strategies (out of several hundreds thousands systems) without ANY manual intervention and then ALL of selected systems where forward tested using SFT (future data). Let me be clear on one thing: I did not cherry picked any strategies.

It seams that every single selected strategy was profitable in the period following the selected generation date. See figures blow:

Test case 1: Strategy creation @ 2014.12.31, Simulated Forward test: 2015.01.01…2016.12.31.
Real-Ticks (Dukascopy data), Real-spread (no commissions)

Test case 2: Strategy creation @ 2016.12.31, Simulated Forward test: 2017.01.01…2018.12.31.
Real-Ticks (Dukascopy data), Real-spread (no commissions)

My conclusions so far:
1. If there are no mistakes, then it seems that it is totally possible to use SQ-X automated ‘custom projects’ to automatically generate and select profitable trading systems.
2. No advanced validation/filtering methods needed. Of course these tests should only improve total result and minimize DD on portfolio level.
3. The results in SFT of >2014 are slightly better than >2016. Workflow is somehow sensitive to used data during strategy generation (due to changing market condition). It seems that years 2017 and 2018 are very difficult years for trading using the selected trading type.
4. It is not 100% proven yet, but it’s a pretty damn good result so far, taking into account it’s based on a simple workflow that is using basic filtering principles.
5. Some of the strategies can be correlated, but for the sake of this investigation no manual correlation filtering has been performed. This would jeopardize the objectivity of this test case.
6. The filtering settings are very rigorous, this workflow filters out only the most robust strategies. According to my statistics only 0.05% of the generated strategies are able to pass this workflow.

TODO:
– Refine the workflow and implement further strategy selection, perform correlation tests, WFM analysis and additional portfolio level related tests.

– I’m also waiting for 01.03.2019 and build 119 with new features 😉

Greets,
Chris

No, you can trade many EA on the same instrument in TS…but I think going short and long at the same time on the same instrument is not allowed (hedging).

The other issue is, it is difficult to distinguish results from different EA’s trading on the same symbol, but there are some ‘tricks’ that make this possible.

Moreover, trading many EA’s = high margin requirements.

open positions will be netted, so how the strategies are trading?

maybe this will help: https://easylanguagemastery.com/trading-multiple-strategies-with-the-same-instrument-part-1/

But…. back to the main topic…..

Few months ago, I’ve claimed to have developed a 100% accurate workflow;) After more than 6months of trading I can provide some feedback on that: “it works” in SQX 😉

HOWEVER! There is a big HOWEVER: it is VERY possible to design a portfolio using SQX, that shows very similar ‘profitable’ behavior in the extended forward data set (your life testing period) as in all previous backtested years that are used for the portfolio design. BUT: This is only true when your data set (=the source of the dataset + trading conditions) are THE SAME, or very similar to the conditions as used during portfolio design. If not, then your live results will be VERY different than what you would expect according to your SQX results.

So conclusion: if your live forex broker data feed is very different from your historical data set (and the same is true for all other trading conditions like spread and slippages), then your results will be also very different. Thus ALWAYS make sure you first check the correlation between your historical dataset and your live data feed. This is nothing new, and already widely known by many experienced traders, but just in case…. it can save you some time and money.

In my case the live results are totally destroyed by SLIPPAGES (and I’m not using very low slippage settings using SQX backtests). There is very low correlation between LIVE trades and SQX backtested trades even on the Dukascopy broker. So far I was not able to synchronize SQX to live 1-to-1, I wonder what correlation other traders get, also other brokers etc..

See the attached figure for a real live test case example (real-ticks real spreads):

Thank you for your sharing, coensio.

It ’s worth reviewing everything you said in this post.

Although I don’t have enough proof, I guess an effective way to reduce the impact of slippage on performance is to use a HT, like 4H, Daily bar.

i dont think so, that slippages has something to do with a trading timeframe – if we are trading stop strategies, we are waiting for a bigger move to hop to the trade – when the bigger move most offen happen?

after news, unexpected events, when something happen and volatility raise

and these times we can get slippage on open trade and this is not timeframe sensitive matter

from my research of my real accounts the slippage is with us, but this is not major issue, mostly the avg. slippage is holding between 1-2 pips for currencies

overtrading could be a problem, because your costs yre raising with every open trade – higher TFs mostly means that number of trades are lower…this is the only difference, not the timeframe itself

also the real tick and real spread is not the best choice…you will get only better backtest, nothing more

Thanks hankeys for your input, but SLIPPAGE or better said the “trade execution” at broker side is the only thing I can think off, to explain those differences…the reason why I’m thinking this is the results of the following LIVE experiment:

1. Make a portfolio in SQX

2. Run it in forward test for few months on DEMO account and also on a LIVE account (the same VPS).

3. Retest portfolio in SQX during the forward period (I still think real-tick real spread is the best option here, at least from what I have seen.)

4. Compare the SQX “forward” backtest to DEMO and LIVE results:

As you can see the results on portfolio level from live trading on DEMO account, more or less correspond to SQX backtested results (= good correlation), however live trading on LIVE account not so much. The main difference between DEMO and LIVE accounts on a MT4 broker is: DEMO accounts are configured by the brokers to have a PERFECT trade execution (perfect timing no slippage), just in order to “push” people to open more live accounts…that is what the brokers do. This particular account made some money in the end but I’m looking for a good correlation between LIVE account and SQX backtest..so far have not been able to reach that goal on a MT4 broker, I see some correlation but I want to see a very good correlation.

Hi,

I’ve experienced same issues than Coensio. I guess slippage is one of the reasons but not 100% sure the only one, I’m suspicious that some indicators could be behind that anomalous behaviour too. Only way I found out to get rid of those strats who behave bad in real is to delete them manually.

However you can’t get rid of slippage, doesn’t matter if your strats are Breakout, the execution time can’t be zero because of the liquidity provider. Not something to blame to the broker. On demo accounts Slippage equals zero because liquidity is infinitum so all the orders can be executed instantaniouslly. Also, slippage will increase with your order volume (big volumes). BUt that’s something will occur doesn’t matter what market/broker you choose. Of course, if you have a shitty broker this problem is worst

Enric wrote:

Only way I found out to get rid of those strats who behave bad in real is to delete them manually

Exactly, this means generating and testing hundreds of strats testing them for a very long time on LIVE and deleting those “bad ones”….and hoping that after this incubation period the strategies will not “expire” very soon…so from a pool of tens I have only few left that correlate to SQX test and show the results “as expected”.

Yes. I see we’ve come to the same conclusion and the number of hours dedicated to control this issue is insane, but as said; don’t see any other option.

However the majority of strats behave acceptable. None of them matches exactly to the backtest, for sure. But most of them have similar trades, what do you think 70% maybe?

Coensio, what is the approximate expected pips/trade value on your strategies?

We trade a relatively big PAMM and we can expect a value of 3 pips for the slippage. I would suggest adding a higher spread value on your data.

In most cases my expected trade is much larger than 10 Pips over 17000 trades.

your strategies or used markets or broker, whatever could be a problem

if i compare my bactests and my real account(s) i am not getting any major differences

slippage is only problem with STOP and MARKET orders – with LIMIT strats not, for limit orders you are getting slippage to your favor

this is comparation of backtest (black) on dukascopy data and RED (real account)

Also in your case I can see that the number of simulated trades and real trades is not equal…so the differences come also from trade execution between SQX an MT4 (not only spread and slippage). But this is indeed an acceptable result….

Which broker is the red line?