Curve fitted strategies and random indicator periods

Hi SQ users and SQ Team,

Just jotting down some thoughts about curve fitting, and issues with optimisation and finding robust strategies. Seems to me, you can end up with what looks like a good strategy with a handful of common indicators (some moving averages, some momentum indicators like RSI and maybe a volatility setting like ATR).

Problem is, even with just a few indicators, all relying on a period settings, is these are just curve fitted strategies. Why should EMA(97) with RSI(29) and ATR(203) for example give great results? Because those values just happen to fit the signals to the historical data. Differences in broker data, and future prices just won’t work.

In the world of trading, common periods used are 5, 10, 14, 50, 200. Seems to me, that unless your strategy is using default or commonly used periods, that might be used other market participents, all you are doing is generating curve fitted strategies.

With this in mind, would it be possible to limit the periods to a list defined by the user in SQ4?

My own findings are that the most robust long term strategies have few differing periods. Those with lots of periods, all unusual and different from each other, are just curve fitted and a waste of time in real trading. Re-optimising them just refits them to data, and is not a solution to the problem.

What do others think?

Cheers,

Mike

Hi Mike,

a good idea if it can be made as a feature, not a “must”. Personally I am finally making some money since the time I am “curve matching” my strategies on 29 years of data and only use strategies with at least 2000+ trades (statistical significance!). Just like before already, they use simple rules only most of the time, not limited by me at all indicator-repertoire wise, but that´s just what SQ finds to work on such a long time-range (most often it´s one of the simple rules as entries then some basic breakout stop sell / buy based on highs / lows, and some basic trailing, most often fixed pips working better than ATR).

However, I am still optimizing the lookback periods for the simple rules afterwards, so going from 20 to 16 for the RSI period for example or from > 50, 40, < 60 for the RSI crossing, via the optimizer, just whatever works better for my fitness-goal. Still, the strategies work so far going forward (finally!), so the huge data-horizon was the trick it seems (most of my previous strategies generated on 15 years "only" and that also failed live, also failed miserably on the 29 years backtest in the years prior 2001), not to limit everything to just basic rules. So personally, I wouldn´t like to see a limitation to such crude steps at all, but as an option, it surely won´t hurt.

Out of interest, do you mind saying which broker the strategies are making money on?

Several, targets are really wide, so brokers don´t matter that much. I am using http://www.globalprime.com.au/forex  though since I know the guys since 2 years and know everything is really executed with banks and know that they use PrimeXM as their execution bridge.

“EMA(97) with RSI(29) and ATR(203)”
This is what Genetic Evolution generally will return with crazy Coefficients like 0.163.
If the strategy fails with slightly adjusted parameters but works amazing with these fine-tuned parameters its dog shit. The optimization step should be higher than 1(I do step:5 for all indicators, they’re all just averages anyway) and higher than step:0.01 for coef. If it still has good results after a “De-optimization” of wide steps in walk-forward, its robust.
 

Depends, I do have such strategies as well and yes, optimized by “1” step-size and they are making money now live forward + I re-optimize them each weekend for a perfect curve fit, BUT ALWAYS on the full 29 years of backtesting data, just with the “last week” added additionally then too. They neither fail the robustness tests (“change indicator parameters by 20% with a 50% chance”, so even worse than the default setting) nor in live forward trading. That was not the case though when I created them on 15 years of data only, then they most often failed already the robustness tests. But the kind of strategies that SQ comes up when creating on 29 years of data have ALL passed each robustness test so far, most likely because the kind of strategies that work on such a long data-horizon are rather simple and are also robust to parameter changes for it´s indicators in a wide-range since they have to exploit a real market edge to work on 29 years of data – not like the kind of strategies that are created on the last few years only that can very well just find inefficiencies in “noise” and “pretend” to be a real edge. Hence I currently see absolutely no reason to not optimize them to the fullest degree possible, which I do every weekend going forward. Whatever makes money is welcome 🙂

You reoptimize every weekend with 29 years of back data for IS? Bored much?

I’ve heard of some people doing it everyday but for HFT.

Anyway you probably know if your strategies are robust or not.
My point is, if changing slippage from 0.5 to 1.0 / spread from 0.5 to 1.0 or 1.5 kills the strategy, its probably over fitted (steps too hardcore) for my taste. On the strategies I run changing a coef by 0.1 or increasing spread or slippage on a backtest will have little visual effect on the equity curve. I attribute this to their robustness.

Yes, every weekend, just takes 1,5 hours per strategy on my rig, nothing problematic nor time-consuming. Staying on track with the market this way. No, as mentioned, not any of the strategies fails when changing or doubling or tripling spread, as well do they all pass the robustness tests including indicator inputs randomization with a 50% chance and 20% change  + work on a randomized data-set that I am using (bootstrapped with a Asirikuy script) – something SQ sadly can´t do at all. As mentioned, none of the 29 years strategies SQ has found so far have failed any of these tests and my guess is it has to do with the simplicity of the strategies it finds that work on 29 years of data, which are few, but if found, great ones:) When I was creating strategies on “only” 15 years of data, many of them failed the robustness tests and also used “strange” combinations of indicators where you could tell this is not an edge it found, but “random noise” it´s trading there – and that got worse the lower the time-range was chosen.

I test pre-2003 for D1 based strategies.
I test after 2003 for intraday strategies.

In 2003 pit sessions were officially closed and markets were officially fully digital. Many say this year also marks a change in markets. A lot of big money was in the pits. Markets are dawning on faster evolution than they’ve ever have but I agree completely that if a strategy is robust through 25+ years, it will probably be for 25 more and forever because its discovering a truth about markets. This is fundamental to my philosophy on trading actually because I know traders much older than I who I admire who have been using the same systems for 30+ years.

2003 does mark a substantive change intraday though.
Here is a PDF study on it:
http://digitalcommons.sacredheart.edu/cgi/viewcontent.cgi?article=1398&context=wcob_fac

“August 28, 2000 (4% share trading on the electronic platform)to September 11, 2003 (share of electronic trading reaching plus-85% on a persistent basis)”.

 

This is when pits were officially “closed”. Some are still active, but are much smaller.

Today almost 100%. This means much more of the money is controlled by algos and not humans. Kevin Davey who has won the Futures World Cup Championship 1 year with 100+% return also uses this year. He also came in second 2 times with 100+% returns. This is a world renown trading championship.

Yes, I hear you, but my strategies are intraday (H1) and most of the ones that were created on 15 years of data only, never worked live going forward. History matters, even prior 2000, also demonstrated nicely in this article which uses 1985 to 2000 data to create systems, and then tests them “OOS” from 2000 to 2015 and they keep on making profits, although the market from 1985 to 2000 was A LOT different than from 2000 to 2015 they make a nice profit in the “computer age trading”, with inefficiencies derived from a age that barely used any powerful computers in trading nor any HFT. That´s exactly the “general” edges that “always existed” that I want to exploit in the market and that have been there for the past 30 years, regardless of other changes in the market like computerization etc., just like your 30+ years manual traders you´ve mentioned.

2 interesting articles in that relation:

http://mechanicalforex.com/2015/12/using-openkantu-in-practice-is-old-forex-data-irrelevant-in-todays-market-an-evidence-based-look.html

http://mechanicalforex.com/2015/12/the-brick-wall-in-trading-system-design-understanding-the-limitations-of-historical-data.html