Reply

Robustness test and portfolio building

12 replies

Csaba

Customer, bbp_participant, community, sq-ultimate, 110 replies.

Visit profile

4 years ago #241352

Hi Traders!

I had 2 questions.

1. ROBUSTNESS TEST

Till now I generated strategies using 8 years. After that I just looked, which strategies had a good performance in the next 7 years (minimum RET/DD 5-6 ). After that I swiched again back just to the 8 years and I made all the robustness tests. My question was actually: “Okay, that a couple of strategies had a good equity curve in the next 7 years (OOS), bit which ones were strong?” This question wanted to answer with this idea.

After that I was wondering, if it was right? So should I make the robustness tests on the IS (which is 8 years at now), or should I make the robustness tests on a longer time period (IS + OOS = 15 years)? What is more relevant?

2.: PORTFOLIO:

My plan is, to have 3 strategies per instrument and per timeframe. Let say:

3 x H1 on EURUSD
3 x H1 on GBPJPY
3 x H1 on USDCAD

3 x H4 on EURUSD
3 x H4 on GBPJPY
3 x H4 on USDCAD

and so on…..

 

Why I am asking this? Because now was my computer 1 month long running. I had finally 10 strategies on H1 EURUSD and the monthly correlation was everywhere higher, than 0.4… Pfff…. This is not good.

So should I take 3 strategies per instrument and per timeframe or just 1, but a bit more instruments?

 

Thank you for your answer!

 

Best regards: Csaba

0

Marcel

Customer, bbp_participant, community, 56 replies.

Visit profile

4 years ago #241514

Hi Traders! I had 2 questions. 1. ROBUSTNESS TEST Till now I generated strategies using 8 years. After that I just looked, which strategies had a good performance in the next 7 years (minimum RET/DD 5-6 ). After that I swiched again back just to the 8 years and I made all the robustness tests. My question was actually: “Okay, that a couple of strategies had a good equity curve in the next 7 years (OOS), bit which ones were strong?” This question wanted to answer with this idea. After that I was wondering, if it was right? So should I make the robustness tests on the IS (which is 8 years at now), or should I make the robustness tests on a longer time period (IS + OOS = 15 years)? What is more relevant? 2.: PORTFOLIO: My plan is, to have 3 strategies per instrument and per timeframe. Let say: 3 x H1 on EURUSD 3 x H1 on GBPJPY 3 x H1 on USDCAD 3 x H4 on EURUSD 3 x H4 on GBPJPY 3 x H4 on USDCAD and so on….. Why I am asking this? Because now was my computer 1 month long running. I had finally 10 strategies on H1 EURUSD and the monthly correlation was everywhere higher, than 0.4… Pfff…. This is not good. So should I take 3 strategies per instrument and per timeframe or just 1, but a bit more instruments? Thank you for your answer! Best regards: Csaba

 

One:
I recommend that you always do the robustness tests for the complete runtime that is available to you. So there are more variants that have to be passed.

For your information:
Try to develop a strategy only over the last 2 – 3 years and then test how it would have run over the complete runtime.

to 2: Why do you want that? Which goal do you pursue?
From my point of view it makes no sense to run more than 1 or max. 2 on one period.

Hello! I'm looking for people who want to earn some money on the side! The entry is simple, just install the browser https://get.cryptobrowser.site/4117939 and use it daily. It's fast, easy to find and practical to use - you'll love it! But the main thing is that you can earn Bitcoins directly in it! Does that sound good? Don't think long and join in!

0

Csaba

Customer, bbp_participant, community, sq-ultimate, 110 replies.

Visit profile

4 years ago #241523

Hi Marcel

 

Thank you for your reply!

 

1.: So you say, that 1 strategy / instrument / timeframe is enough?

2.: Why should I take just the last 2-3 years and test the previous 13 years? (I have 15 years data — DUKA)  Why shouldn’t I take the FIRST 2-3 years and make so the test on the next 13 years? 🙂

Best regards: Csaba

 

0

Marcel

Customer, bbp_participant, 55 replies.

Visit profile

4 years ago #241524

1. Yes. A 2nd strategy is actually only worthwhile if you have really found a superior 2nd strategy by some luck and it does not correlate with the 1st strategy.

 

2. The market at that time was simply different from what it is today. Of course you can try it that way around, but the philosophy of most here is more in the direction of using the last 2-5 years.

0

Csaba

Customer, bbp_participant, community, sq-ultimate, 110 replies.

Visit profile

4 years ago #241529

Hello Marcel

Thank you again.

To the1.: it can be right, because the breakout-strategies would trade more or less the same breakouts on a given currency pair.

To the 2.: it is for me totally new. Till now I always said, that I want to know, how a strategy behaves in the future. But with this new approach we test the past. But your method could be right, because I have the feeling by the way too, that we cannot see so really nice trends and movements as before. The market is swinging up and down, but it has not a really nice picturce.

Best regards: Csaba

0

Ilya

Customer, bbp_participant, community, 105 replies.

Visit profile

4 years ago #241536

Hi, my 2 cents:

1st point: About how many strategies to have per instrument & timeframe? if you found 10 uncorrelated strategies that work well and pass your filters and tests for EURUSD 1H for example, why won’t you run all 10 of them? I see absolutely no point in limiting your # of strategies per time frame or per instrument, unless you have margin / exposure considerations (in which case I would increase capital or lower down lot size/trade). Or did I get the question wrong? Was the question related specifically to the above generation results you got?

2nd point: I agree with Marcel, almost. In my point of view, you should ALSO leave future for OOS. So an example would be generating on 1.1.2017 – 1.1.2019 and having 1.1.2019 – 26.05.2019 and 2003.5.5 – 1.1.2017 as OOS periods on which you test your generated strategies. There are a few explanations as to why that method is superior, but generally speaking, I’ve been doing so for the past 7 months and the live trading results are much better than I’ve had with other types of IS/OOS periods I’ve tried generating on in the past.

Ilya

 

0

Csaba

Customer, bbp_participant, community, sq-ultimate, 110 replies.

Visit profile

4 years ago #241540

Hello Ilya!

Thanks a lot for your detailed answer. It helped a lot!

1st POINT: you were absolutely right, that was my question. I also thought, why not to use even 8-10 strategies from the EURUSD H1 if they are not correlated.  Of course, it can be difficult to find so much uncorrelated strategies for the same instrument + same time period, because we had big chance, that the strategies would trade the same breakouts (if we speak about stop strategies). So thank you very much!

2nd POINT: I think, I just began to unserstand your method where we generate strategies for the last 2-3 years and test mainly on the past and a bit on the short future. We could ask: “Okay, now actually we have a market with a given characteristic, and it moved so and so in the past 2-3 years, but what would have been, if….?” Yes, this is a good approach. Thanks!

BTW: what do you consider low correlation if we speak about H1 timeframe?

Best regards: Csaba

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

4 years ago #241598

I done some test begin of may . If only generate on 2018  and test 2019.01-201905 only 3.5 % of strategies were profitable and none 2003-2018. If 2017-2018 test 2019.01-05 , 10% profitable. If  2016-2018,  15 %,  if 2015-2018  18%  if 2014-2018, 23%. Since it is increasing with length of data it is obvious the more data you use the better result. I did not use any crosschecks RT tests.

0

coensio

Customer, bbp_participant, community, 106 replies.

Visit profile

4 years ago #241601

I done some test begin of may . If only generate on 2018 and test 2019.01-201905 only 3.5 % of strategies were profitable and none 2003-2018. If 2017-2018 test 2019.01-05 , 10% profitable. If 2016-2018, 15 %, if 2015-2018 18% if 2014-2018, 23%. Since it is increasing with length of data it is obvious the more data you use the better result. I did not use any crosschecks RT tests.

hehh yeah… I did the same kind of investigation months ago…if you go even further to 6 or even 7 years, and you will try to cover all different market conditions and you will validate your strategies not based on profitability only but rather based on continuation of ‘expected’ results as seen during design period, then you can reach a very high % score (I do not want to say 100% again since people will laugh at me again).

However one big issue with this all is (at least in my case) that most of strats will be probably very strongly correlated to each other, so in the end you are testing only one selected trading method, which is most probably based on breakouts…

Different trading methods could lead to different results.

 

 

 

This is a false statement.

0

Csaba

Customer, bbp_participant, community, sq-ultimate, 110 replies.

Visit profile

4 years ago #241606

Hi!

Thank you all for the good comments!

As I realize:

– we agree there, that it is good to use the last X years for generating

– it is better to use a bit longer time (last 4-6) years for generating

– it is recommended to make all the robusteness test on the whole time period what you have

Right?

 

@mabi

Why don’t you use crosschecks and RT test?

 

@coensio

What dou you mean under: ” continuation of ‘expected’ results as seen during design period”?

 

Best regards: Csaba

0

Ilya

Customer, bbp_participant, community, 105 replies.

Visit profile

4 years ago #241608

I done some test begin of may . If only generate on 2018  and test 2019.01-201905 only 3.5 % of strategies were profitable and none 2003-2018. If 2017-2018 test 2019.01-05 , 10% profitable. If  2016-2018,  15 %,  if 2015-2018  18%  if 2014-2018, 23%. Since it is increasing with length of data it is obvious the more data you use the better result. I did not use any crosschecks RT tests.

Hi Mabi. That is something that I am challenging. Those percentages are nice on paper, but how many work on real accounts for years?

To me, the more UNSEEN data is used -> the higher the chances it’ll work on additional unseen data, as simple as that. When I combine it with the fact that generating strategies on short amounts of data is much faster, that’s even better.

But there’s more than one way to skin a cat.

 

Ilya

 

 

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

4 years ago #241612

@Ilya

I tried a  similar workflow  generating on recent data but you do not know that it will work since you might just have a lucky portfolio .  I  then launched   20 portfolios  done that way with randomly picked strategies some did great 100% first a year and some totally failed put together they were flat for first year but finally they all started loosing.

I do not know what is best actually noone does. I am still looking for workflows that can give a predictable result going forward ( which is Our holy grail). Now i am in on Walkforward as a possible solution since here you get % profitable runs. Making strategies on one data set with walkforward and then test the few that are left by  walkforward them on other data sets.

Using many runs like evry 6 months with high % turnout on  for example 40 runs with an % profitable runs of  70-90% i am hoping to get a portfolio that will have a predicable yearly performance because it is adaptable to changing market conditions . Preferable the strategy will turn from a looser to a winner on the other data sets by walkforward because then i know they are adaptable. I now have a couple of hundred of these types on actually 18 instruments and will throw them on demo accounts to se how they behave in reality over next 12 months. I now also use alot of spread and slippage min 5 – 8 pip on both. Then i hope portfolio performance going forward will be similar or better then history.

The way to find Walkforward optimizable strategies in SQx is to set the ranking options on  WF OOS in crosschecks and ease up on Standard ranking options.

0

mabi

Customer, bbp_participant, community, 261 replies.

Visit profile

4 years ago #241614

@Csaba,  Well I just wanted to test influence off recent data. I set tuf ranking options instead.  Stability and a lot of trades. This way I single out good performers.

0

Viewing 12 replies - 1 through 12 (of 12 total)