Reply

Robustness tests for what?

5 replies

Csaba

Customer, bbp_participant, community, sq-ultimate, 110 replies.

Visit profile

4 years ago #245556

Hi Friends!

I made an interesting experiment.

I had 120 strategies, that seemed to be very good from 2004.01.01. – 2017.01.01.
From these 120 strategies:

– 20 were nice between 2017.01.01. – 2019.01.01. too
– 100 have been failed. The equity curve went down.

What I wanted to know: how do look the robustness tests at the 20 good and the 100 bad strategies? (For me are the DIFFERENT MARKET and the RANDOMIZE PARAMETERS the most important. I made now just these 2 tests.)

The result is very-very interesting!

I found, that:

– none of the 20 good strategies passed the diff. market and rand. parameters
– 6 from the 100 bad strategies passed both the diff. market and rand. parameters

So I would have taken just loser strategies if I decide based on the robustness tests.

Question: ROBUSTENSS TEST for what?

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

4 years ago #245565

I believe using robustness tests is only away to sort out strategies based on their worst historical performance hoping that the future result will not be worse then that. It cant in anyway tell you what next year expected performance is it can only simulate what it can be in worse or best case and that is long term short term you have to look at things as max draw down. So if the max simulated draw down is 100% you can expect 100% draw down in worst case when you start trading it immediately  :). So to counter this you have to build a diversified portfolio.

0

Marcel

Customer, bbp_participant, 55 replies.

Visit profile

4 years ago #245581

Hi Friends! I made an interesting experiment. I had 120 strategies, that seemed to be very good from 2004.01.01. – 2017.01.01. From these 120 strategies: – 20 were nice between 2017.01.01. – 2019.01.01. too – 100 have been failed. The equity curve went down. What I wanted to know: how do look the robustness tests at the 20 good and the 100 bad strategies? (For me are the DIFFERENT MARKET and the RANDOMIZE PARAMETERS the most important. I made now just these 2 tests.) The result is very-very interesting! I found, that: – none of the 20 good strategies passed the diff. market and rand. parameters – 6 from the 100 bad strategies passed both the diff. market and rand. parameters So I would have taken just loser strategies if I decide based on the robustness tests. Question: ROBUSTENSS TEST for what?

 

Here it would be interesting to know what kind of robustness tests you mean or did.

If you assume that the robustness tests are possibly unnecessary, you also have to tell us what kind of test you are talking about.

 

0

tomas262

Administrator, sq-ultimate, 2 replies.

Visit profile

4 years ago #245584

If the robustness test has negative impact on strategy performance it does not have to necessarily mean the strategy is bad and must be thrown away intermediately.

Does the logic behind what makes the strategy work make sense? If so and if the strategy continues to work after many years of backtest data you could consider keeping it. But it also depends which of RT tests went bad whether it is the slippage test, parameters change test, just trade order modification etc. So you need to investigate what caused the strategy RTest to fail and how likely is that the worst case scenario shown by the robustness test comes true in live trading

0

Csaba

Customer, bbp_participant, community, sq-ultimate, 110 replies.

Visit profile

4 years ago #245587

Hello Everybody!

Thank you all for the answers. You asked, what kind of RT I made / mean. I wrote in my post: “For me are the DIFFERENT MARKET and the RANDOMIZE PARAMETERS the most important. I made now just these 2 tests.”

Of course I make also slippage, randomize orders tests, etc., but the above mentioned 2 tests are for me really important. And for this little experiment I made just diff market and rand. parameters. For me these 2 tests are the most crucial.

And for you?

0

Marcel

Customer, bbp_participant, 55 replies.

Visit profile

4 years ago #245588

For me are the most important tests:

 

1.  Slippage

2. Different Timeframe

3. Different market

 

0

Viewing 5 replies - 1 through 5 (of 5 total)