Robustness Test Failed?

Hi, 

 

I’ve purchased the software some weeks ago and now I m trying few things out. 

 

At the moment I could already find some strategies which seems to be in profit for EUR/USD, and also after I tested this on GBP/USD it still looks good. But some of them completely fail in the robustness test.

 

This is what I have done:

 

1.) Build strategy on EUR/USD for a period with data of over 5 years (1H strategy, but tested on 1min) => OK

2.) Tested in on GBP/USD => OK

3.) Improved it for a period of 2.5 years (in the sample) and other 2.5 years (out of sample) => OK

4.) Retested it on EUR/USD and also GBP/USD again => OK (sometimes GBP/USD is even a little bit better)

5.) Robustness test => FAILED

 

Failed in my opinion because of high Drawdown percentage (>25%) and very low DD Ratio (<1.5)… also the graphs looks very scary.. the original strategy is going up, and all other lines are somewhere else…

 

Now my question is… how important is the Robustness Test? My strategy looks to work on both currency pairs and also looks good on the right side of the chart in the out of sample period.. Does the robustness result now mean that it would fail in real life?

 

Just trying to get a feeling about the difference of theory and real world…

 

Regards,

Geffect

Hi Geffect,

I think the robustnesstest is very importend.

 

If a strategy fail (In my opinion, a strategy fail if Netto Result is low), I don´t use the strategy.

 

I do the following.

 

1) Strategy generation with ISS and OSS

2)Forwardtest with unknown data

3) Rebustness test

 

If a strategy only fullfill 1) till 3) I will use the strategy on demoaccount.

 

thomas

yes, I agree. Robustness test is very important.

 

If your strategy fails because it cannot cope with slightly changed parameters, or missed trades, etc.

then it is not robust enough. It would very probably fail also in live trading.

 

Mark

Thanks to Tnickel and Mark for the answers.

 

How do you decide if the robustness test failed or not?

 

At the moment I m looking for 2 things at the robustness test result:

 

1.) Maximum Drawdown at 95% level is about the same like the original result or slightly more… 

 

Example: 

original 7% and robustness result at level 95 is 10% ===>> OK

original 7% and robustness result at level 95 is 18% ===>> not OK

 

 

2.) Ret/DD ratio is over 1.5 at 95%

 

Does this makes sense? Or do I miss something important or my numbers are not realistic?

 

Regards

Daniel

I just ran my first test following the Article Mark posted (Following Building EURUSD).  At the 4th Filter, I got many good RT results.  

 

So exported them into Excel and compared them based upon the percentage difference between the strategies results and the RT’s results.  

 

In excel, I was able to filter out (using the Filter feature in the Sort button).  I only showed the strategies that are <25% difference.  I was left with a manageable list where the RT results did not vary by more than 25% than the strategies' results.  

 

I think excel helps to mess with and analyze the data.

 

-Stearno