Fragile “Test precision”

Hello,

 

I wish to get answer does anyone has also same problems with too fragile “Test precison” methods.

 

I have data from 9 brokers (tick data) + Dukascopy. 

 

 

 

 

When I generate strategies by:

 

1. Real Tick method – then tests take much longer time, but seems they are most accurate. 

 

2. 1minute method – then test take very short time, but after passing strategies to “Retest” section and run under “Real Tick” – all gets bad and none of strategy has positive NetProfit. 

 

 

All of settings remain the same. In passing strategies to “Retest” I pick to move also all settings. Then I only change Test Precision to Real Tick. 

 

 

Has anyone got solution? Because in this way, there is no any reason to use 1 minute method in any reason.

It depends also how tight SL and TP you use and how many pips is average trade.

The bigger values you will use, the better results you will get.

 

Patrick

I don’t know if its sufficient, but currently I’m trying with H1 timeframe and TP/SL with ATR Multiple from 8 to 25 and still get same results – on 1 min it’s ok, but on Real tick all fail. 

it depends also on the type of your strategy – if strategy is opening consecutive orders (one order opens immediately after the previous one closes)

then the slightest changes in time when the order was closed could have big impact on results.

 

This is also a signal of non-robust strategy.

You should throw away strategies that don’t have same or very similar results on tick and minute data.

Yes Mark, I know it, but I see a very fragile metodology here.

 

Eg. Strategy with 200 trades among 4 years generates diffirent (Very diffirent) results when testing on TICK method with spread 1.5 , and when testing on spread 1.7 . One seems to get profit, second losses. 

Other thing if I throw this strategy onto “real tick” method . Then I get very, very bad results. 

 

And believe me, I’m not trying to make “scalping” strategy, but quite long term MA strategy with some wide TP/SL based on ATR Multiple. (from 3 to 8)

it really depends on strategy. If your strategy opens consecutive orders then spread bigger than 0.2 pips could have a big influence.

 

For strategy whose trades are not connected to each other this shouldn’t matter that much.

 

So in a way it is also a test of robustness – if your strategy has totally different results when spread is changed only slightly it is a clear sign it was curve fitted to data.