MT4 backtest wrong results

Hi,

I hope it’s not a too-difficult question, but I have been struggling with this problem for a few days straight now and it is holding me back, I hope someone can assist..

I’ve generated an EA on SQ3 which gave good stables results on 15 years of Data, robustness etc. Before going live, and after a small WFM, I am trying to run a retest on MT4, but the results are very different. I run both backtests on dukascopy tick data, same date range (I tried the range of 1.09.2012 – 1.09.2018 for backtest verification), and the timezone is correct (since other EAs seem to yield identical results).

in SQ3, this date range yields 240 trades, while mt4 yield 55 trades. Journal doesn’t seem to have any errors, “results” tab seem to indeed skip trades, some orders are opened but aren’t executed (I’ve tried with 0 spread also so that’s not the issue, since it’s the exact same data), some orders stay pending much longer than in SQ3 backtest and some just aren’t opened. Some of them are fine though. Going through the code, I’ve tried to change “modifyinsteadofrreplacing” to false (Since there’s an exit rule that states exit after 31 bars, and modification doesn’t change that initial rule), but that didn’t help.

I’m lost as to what’s the lack of correspondence here.. yes this EA isn’t the most simple “MA cross” EA, it uses stoch, pivots, ichimoku etc, but since SQ works on the mt4 engine, I’d expect the results to coincide..

I’m attaching a zip that contains the STR, MQL4 and the code itself in a text file.. with the hope that someone has the will to take a look.

Much appreciated

Hi Notch, thanks for the reply and the report. As I can see, again there are 70 trades, in comparison with 240+ with SQ3.. Not sure what to do, if I should run it live on micro lots even with this very high non-correspondence or trash it..

Nope, I did  WF matrix with 3 years data, optimising 6 main parameters with these results.

Thanks for sharing

I backtested it using all available data (my preference)

I have no opportunity to test it tith SQ3 now, but I did it on SQX

MT4 results differ (maybe because sqx is using sqx indicators and strategy was made tith SQ3 indicators)

Overall this strategy is not bad, I would consider adding it in to a big portfolio of algos, but I would never let this strategy trade my capital alone

It completely failed monte carlo for 15 years in SQX, but hard to tell (as I said it could be indicator differences)

BTW

with risk of 1% it looks a bit better

Vaidotas

Vaidotas Segenis wrote:

Thanks for sharing I backtested it using all available data (my preference) I have no opportunity to test it tith SQ3 now, but I did it on SQX MT4 results differ (maybe because sqx is using sqx indicators and strategy was made tith SQ3 indicators) Overall this strategy is not bad, I would consider adding it in to a big portfolio of algos, but I would never let this strategy trade my capital alone It completely failed monte carlo for 15 years in SQX, but hard to tell (as I said it could be indicator differences) BTW with risk of 1% it looks a bit better Vaidotas

Hi Vaidotas, thanks for giving a hand. My results are very different though. The entire data has 519 trades (4 times the amount you got), and has the MC results attached (200 runs of randomize strategy parameters with probability 20% and max change 50%), as well as very good results on other MC tests. Equity curve is very different too.

That’s why I could not understand the huge lack of correspondence of the MT4 retest and SQ3. I thought there is obviously a bugged parameter or one of the rules is mutated upon generation of the MQL4, that’s why I started the thread in the first place.. in the form it is now after exporting, I think I would actually leave it out of the portfolio. I would add it only if I can find the reason for the huge mutation and make it seem like my SQ3 result..

Was your SQX result based on the STR file? was it done with all data? since if so, and there actually is so little trades, it’s actually a problem with SQ3, and not the MQL4 file generation as I thought.. I don’t have a way to try SQX now unfortunately yet.

Thanks again for helping me figure this out.

Hi you can open an errorreport.

They look for every error in this error- trackingtool

https://roadmap.strategyquant.com/projects/sq4/tasks/new

thomas

Ilya wrote:

I think I would actually leave it out of the portfolio

No worries. I’ll give the strategy to a friend so it doesn’t go to waste.  

Feel free 🙂

Ilya

Ilya wrote:

Vaidotas Segenis wrote:

Thanks for sharing I backtested it using all available data (my preference) I have no opportunity to test it tith SQ3 now, but I did it on SQX MT4 results differ (maybe because sqx is using sqx indicators and strategy was made tith SQ3 indicators) Overall this strategy is not bad, I would consider adding it in to a big portfolio of algos, but I would never let this strategy trade my capital alone It completely failed monte carlo for 15 years in SQX, but hard to tell (as I said it could be indicator differences) BTW with risk of 1% it looks a bit better Vaidotas

Hi Vaidotas, thanks for giving a hand. My results are very different though. The entire data has 519 trades (4 times the amount you got), and has the MC results attached (200 runs of randomize strategy parameters with probability 20% and max change 50%), as well as very good results on other MC tests. Equity curve is very different too. That’s why I could not understand the huge lack of correspondence of the MT4 retest and SQ3. I thought there is obviously a bugged parameter or one of the rules is mutated upon generation of the MQL4, that’s why I started the thread in the first place.. in the form it is now after exporting, I think I would actually leave it out of the portfolio. I would add it only if I can find the reason for the huge mutation and make it seem like my SQ3 result.. Was your SQX result based on the STR file? was it done with all data? since if so, and there actually is so little trades, it’s actually a problem with SQ3, and not the MQL4 file generation as I thought.. I don’t have a way to try SQX now unfortunately yet. Thanks again for helping me figure this out.

Yes I used str file with all the data, sqx and mt4 difference is not huge

dont use Pivots in the strategies – they are very sensitive to data and on every broker (or backtester) you will get diff results – thats my findings

hankeys wrote:

dont use Pivots in the strategies – they are very sensitive to data and on every broker (or backtester) you will get diff results – thats my findings

Yeah I had that thought.. for some reason Pivots are dominating in good strategies in my generation attempts on SQ3.. Will try without.

Thanks

cheers

Vaidotas Segenis wrote:

Yes I used str file with all the data, sqx and mt4 difference is not huge

Thanks, that’s good to know, though a bit disappointing that SQ3 can produce such inconsistencies. I hope SQX will be more stable in that aspect.

Cheers

Yes, it is very important to have a correct time zone

I always clone my data to utc+2 in SQX and TDS

However I do not believe timezone difference can have such huge difference like in your case Ilya.

Something else is wrong with your sq3 data I believe. I agree with notch

Do you think there could be a problem with how you uploaded the data in SQ3?

I attach my equity curve and backtest result on all data, the data tab for the 01.09.2012 – 01.09.2018 run (along with that backtest result) and my data manager..

Ilya

And as another follow up, I attach the optimised strategy (coefficients are optimised, and I re-ran all robustness tests to make sure they’re good still), along with the results & Equity curve of it on SQ3.. If anyone helps me to figure out what I’m doing wrong, I’ll be forever grateful