Assessing if a live EA needs to be shutdown

Hi 

I was wondering, if a couple of strategies that perform well on backtest specially out of sample and in live trading start to perform poorly,what do you do?

do you shut them down? do you keep trading them at small sizes? put them in a demo and wait for them do show better results?

what criteria do you use to assess if a strategy is broken or not?

a possible way to do it is do do a monte carlo on the historical trades and check in which quantile the current live trades are at.

Anything below quantile 0.2 I would become suspicious

you determine if the strategy is dead by looking at the MonteCarlo’s 95% Confidence level,

if your backtest (IS/OOS) was for example 10% DRAWDOWN and the MC’s 95% CL show’s you 25% <<< Expect 25% of DRAWDOWN,

other then that you should remember your consecutive losses/wins + winning% + profit factor + recovery factor + and other indicators that could indicate you if you’r strategy acting oddly and not as it should..

good luck and read more about the subject.

i would recommend you to check out this blog and read all the posts..: http://www.longtermforex.com/

Many Thanks Karish! , there are some limitations with MCs if there is a certain auto-correlation in the trades they will get broken

some other creative approaches: I’ve seen , trading the equity curve with an EMA crossovers, problem with that is that it can lead to whipsaws, when the curve is flat

it is more complicated. you should find out why is the EA not working. is it because its range trading EA and the pair is trending at the moment? or is it because the volatility on the pair is too high/too small? or is it because it had many winning trades already this year? or is the strategy spread sensitive? or is it because the source code of this strategy is not logic? or may be the robustness of the strategy is not “storng enough” / or backtesting period was too short?

if you find out what’s wrong, you can find out how to do it right. 

the safest option is always: 1. demo forward test 2. live test with small trade size 3. live trading. But not everybody is so patient.

Many Thanks Patrick, those are valid points to consider and a wise way to proceed with regards to putting real money to work

I suppose one could use a couple of benchmark EA, simple EAs :one for Trends, another for range and another for Volatility and see how the EA is performing against the benchmarks

Market movement is also another factor. Taking for example these recent days, Market has been pretty undecisive with a lot of false breakouts, hence strategies that were generated based on stop orders will likely take the hit while strategies that are using limit orders might have performed better. And when the market moves out of this undecisive zone and starts trending, the table will be turned. So I think its good to observe the market condition you’re in versus the type of strategy you are using. It might not be the case that your strategy is no longer effective but rather the current market condition is not favorable for your strategy. So i think it requires some patience to let your strategy ride thru the difficult period.

thanks daveng, I guess one approach would be to have a couple of simples EAs, one for breakouts another one for ranges , that are to be used as benchmarks

thanks daveng, I guess one approach would be to have a couple of simples EAs, one for breakouts another one for ranges , that are to be used as benchmarks

Indeed…good that in SQ it allows you to focus on generating breakout or ranging strategies.

I record a couple of things, but the heart of knowing when to stop it (for me) is comparing the results from strategy generation and OOS testing for last rolling 12 months and 30 trades incubation  vs. live performance.

So, I have 3 time frames from above samples to compare real performance too.

What I’m recording and looking at are win loss % and consecutive losers, that’s it. Super simple. If there’s big (bad) variation from real live trading from any of the 3 recorded data points, I stop it. This part is fairly subjective (the variance I look for, not what I’m looking for).

Should be pointed out, putting a strategy from generation to OOS testing to incubation, I’m not looking at (just) win loss % and consecutive losers, but those are what I’m recording to determine when I stop the strategy. I’m also doing other things as it works through my process, like Monte Carlo, etc…but specifically for when to remove a live trading strategy, that’s what I do. What I’m looking at to determine if it goes from generation to OOS to incubation, are fairly standard, Sharpe, ret/DD, etc..etc..but to know when to pull it, it’s just win loss % and consecutive losers. I already know the strategy works and all the pieces, stops, profit targets, etc. are ok since I incubate it for 30 trades, live data, etc. before it goes live (IF, big if).

Ideally, you should know when to stop a strategy before you put it live, so there’s nothing subjective. It helps to remove uncertainty.

Also, I don’t use MT, I use NT, but I guess it’d be a similar sort of process.