New strategy best way to rank?

Hi, Could others share what results they use to rank a new strategy?

Sorting by net profit isn’t always the best.

Is profit factor the most important? Or return/drawdown ratio maybe?

Thanks.

Custom fitness. I generally run around this, sometimes with slight adjustments depending on the pair’s results and if I’m making a trend following strategy or a mean-reversion strategy-

Net profit- 0.8
Max DD %- 1 (Money management always % risk therefore max dd %)
Stability- 1
Stagnation %- 1

This will give you a pretty smooth equity curve when the fitness starts reaching around 0.65 or higher.

Thanks Threshold and also thanks for your useful videos.

As a newbie I find there is a lack of a clear thorough manual to explain all the settings.

I even bought the ebook which helped with the whole process but again lacked explanation.

It also missed out the “Improve Strategies” tab altogether.

Any more info appreciated.

Basically I follow the guidance from the ebook written by Zdenek  Zanka, “How to Trade Profitably in Forex Using StrategyQuant Software” and made some tweaking here and there.

 

Even with the selection criteria, there will still be several hundreds strategies generated.

 

I generally would use the following parameters for my strategy ranking / selection :

– Net Profit

– Stagnation

– Max % Drawdown

– Return / DD Ratio

 

The key is to keep things Simple,

 

 

 

Body like a Mountain

Heart Like an Ocean

Mind Like the Sky 

 

in harmony with the Universe and trade like “water”, 

I use a custom ranking.

 

Ret/DD 1

Stability 5 to 10

Stagnation % 1

 

I am interested in the most linear equity curve possible.

 

PS.  I always select a date and end date for the data where the price is roughly the same start to end, so that there’s no long or short bias, and discard strategies that are not long/short 50% of the time.

I use custom ranking. I forget exactly the weighting, but essentially, I’m looking for win %, profit factor, minimize losses and bigger winners, small DD with a nice predictable stability.

 

Make sure you set the 3 part validation – IS training, IS validation and OOS. Then set so there is a % similar (I use a band of 80% to 120%) between IS and OOS, so the win % is roughly the same. That weeds out a lot of strategies that fail OOS and have bad stability between IS and OOS.

 

I’m not really looking specifically for profit over a given time period that I generate the strategy (but profit factor is pretty high up there), because I’m not going to run the strategies on a historical time frame, so I want to find that it identifies winners, cuts losers short, that sort of thing, because the conditions I run it on will be different than the historical conditions I generate it on.

 

And this below, really this read lots of material and keep it super simple resist the urge to make super complex and highly demanding criteria…. I may want a Sharpe ratio of 3, but so what if I get it on data from 2003 in a historical test. What I really want is something that can predict 80% winners in 2003 and 2016 and identify and cut losers fast.

 

Basically I follow the guidance from the ebook written by Zdenek  Zanka, “How to Trade Profitably in Forex Using StrategyQuant Software” and made some tweaking here and there.

 

Even with the selection criteria, there will still be several hundreds strategies generated.

 

I generally would use the following parameters for my strategy ranking / selection :

– Net Profit

– Stagnation

– Max % Drawdown

– Return / DD Ratio

 

The key is to keep things Simple,

 

 

 

Body like a Mountain

Heart Like an Ocean

Mind Like the Sky 

 

in harmony with the Universe and trade like “water”, 

Make sure you set the 3 part validation – IS training, IS validation and OOS. Then set so there is a % similar (I use a band of 80% to 120%) between IS and OOS, so the win % is roughly the same. That weeds out a lot of strategies that fail OOS and have bad stability between IS and OOS.

 

Hi _Cujo!

 

Would you like to further explain the IS validation part?
My ranking settings looks as in the screenshot attached. Am I missing something? Do you suggest to verify something with IS V also? When sorting through Views in Databank or?

Just generating on high Win Rate %, Stability, small DD and positive Profit Factor is something I’m gonna try also. Could you generate any robust ones with spectacular win rates like >80% ?

Like this..it’s not actually a ranking (but I use custom for ranking), it’s a filter condition for the strategies going in to the databank…

 

 

I’m not totally happy with it, I’m thinking about ditching the Max DD % requirement (but keeping the % wins, possibly tightening it up even to 90 to 110%, although I just toghtened it up a few weeks ago). The Max DD% can be impacted by the length of time. Although, not so relevant for “many” years of data, it’s relevant for short term OOS, like if you OOS test for 6 months, or something, it’s better to use Ret/DD ratio, I just haven’t gotten around to changing it yet, as I try to set and forget SQ  (but keep a list of changes to make every few weeks or so), and I just haven’t yet.

 

Anyway, rambling aside, my thinking on this, is…

I don’t actually care what things like Sharpe Ratio, SQN, R expectancy, etc..etc… was 10 or 15 years ago. I’m generating ~15 years or so of data (futures, not fx).

 

I care about if it’s consistent. Is it consistent between training, validation (and also OOS and then going forward from today, in real trading, ofc but that’s further down the line, this part here is at the very start in data bank filtering). I forget exactly, but with 10-15 years of data generating, I think training and validation are each about 5-7 years. So, if it has a fairly consistent win % between 2 separate batches of ~5-7 years each, it’s a good start to consistency…

 

For example, if a strategy had a 80% win rate in the IS training, and a 30% win rate in the IS validation, it’s probably garbage, even it it meets my other requirements (like profit factor, payout ratio, etc..etc.. DD%, etc..) because it’s not consistent. Why would it then be consistent today? I found it better to specify this sort of ratio since, ofc  different types of strategies have different win % depending on the type of strategy, and I got tired of changing data bank filters all the time…eheh… but if it’s consistent, that’s what I want.

 

I have a bunch of other stuff in databank filters too, ofc, …this is just a part of it.

Hi _Cujo!

 

Would you like to further explain the IS validation part?
My ranking settings looks as in the screenshot attached. Am I missing something? Do you suggest to verify something with IS V also? When sorting through Views in Databank or?

Just generating on high Win Rate %, Stability, small DD and positive Profit Factor is something I’m gonna try also. Could you generate any robust ones with spectacular win rates like >80% ?

 

 

…actually looking at your screen shot (I didn’t last post)…I wouldn’t do it like that.

 

You’re basically telling it you absolutely really, really care most about profit at some point in the past. I think it’s a personal thing, but…YES, it’s important it made money in the past, as you want other things to be working (like minimizing DD,  etc.. which will be indicated by it making money), but JUST looking for much for making money in the past, might mean there was 1 or 2 big trades it caught, and it falls apart if it misses them.

 

Would de emphasis the actual profit (but still keep it, as long as it makes profit, but consider using profit factor instead, or in addition to pure $), and pump up emphasis on the attributes that would lead to profit, like win %, minimize losses, payout ration, DD, stability, etc.. some of the other aspects you have there, as that’ll help weed out the strategies that maybe caught a lucky  trade, but are otherwise fragile garbage. For example, if it can’t cut losers, and let winners run, it’ll get weeded out by payout ratio, if it can’t actually identify direction, it’ll have a crap win % (unless it’s trend following, in which case it’ll have crap win %, and good payout), etc..etc…so, look for the things that will lead to profit, not the profit itself.

 

Because, ultimately, you don’t care if it made money in the past, you want it to make money tomorrow, which it’ll do if it has an edge (and is identified by your criteria), and the edge isn’t about the profit, per see, profit is a result of the edge, if that makes any sense, or just rambling nonsense, which it could be, so don’t take my word for it….

…actually looking at your screen shot (I didn’t last post)…I wouldn’t do it like that.

 

You’re basically telling it you absolutely really, really care most about profit at some point in the past. I think it’s a personal thing, but…YES, it’s important it made money in the past, as you want other things to be working (like minimizing DD,  etc.. which will be indicated by it making money), but JUST looking for much for making money in the past, might mean there was 1 or 2 big trades it caught, and it falls apart if it misses them.

 

Would de emphasis the actual profit (but still keep it, as long as it makes profit, but consider using profit factor instead, or in addition to pure $), and pump up emphasis on the attributes that would lead to profit, like win %, minimize losses, payout ration, DD, stability, etc.. some of the other aspects you have there, as that’ll help weed out the strategies that maybe caught a lucky  trade, but are otherwise fragile garbage. For example, if it can’t cut losers, and let winners run, it’ll get weeded out by payout ratio, if it can’t actually identify direction, it’ll have a crap win % (unless it’s trend following, in which case it’ll have crap win %, and good payout), etc..etc…so, look for the things that will lead to profit, not the profit itself.

 

Because, ultimately, you don’t care if it made money in the past, you want it to make money tomorrow, which it’ll do if it has an edge (and is identified by your criteria), and the edge isn’t about the profit, per see, profit is a result of the edge, if that makes any sense, or just rambling nonsense, which it could be, so don’t take my word for it….

 

Get your point totally.

But, I don’t have anything in my criteria which measures absolute profit numbers, right? Did you mean profit factor is a bad criteria to use?

Get your point totally.

But, I don’t have anything in my criteria which measures absolute profit numbers, right? Did you mean profit factor is a bad criteria to use?

 

Net profit weight of 5 in custom fitness.

 

For sure use profit factor, it’s better than simple net profit.

 

Having said that, you might want to think about having just a flat 2k profit filter too, that’s in the ebook, forget over what time frame the 2k is. That’ll ensure marginal strategies don’t make it to your data bank, but profit factor of 1.3 should do the trick too, as you have.

Net profit weight of 5 in custom fitness.

 

For sure use profit factor, it’s better than simple net profit.

 

Having said that, you might want to think about having just a flat 2k profit filter too, that’s in the ebook, forget over what time frame the 2k is. That’ll ensure marginal strategies don’t make it to your data bank, but profit factor of 1.3 should do the trick too, as you have.

 

Ah of course! Thanks for pointing it out for me _Cujo!
I remove Net Profit from the weight and give Profit Factor and Win Rate % high weight instead.
I’m doing “Divide In Sample period to Training / Validation” in Genetic Options now also.

 

Edit: Heh, was no Win Rate % in custom fitness.