Tools to find a filteringprozess
6 years ago #113664
has some tried to find profitable strategies with datamining tools?
For example, I am looking for a profitable filteringprocess.
1) First Step
Generation of 20000 Strategies with the SQ
I want to filter the profitable Strategies out of 20000
With datamining tools I can find a profitable filtering process.
Who has knowledge in this topic ?
Why would you need to use such an external tool while SQ has all filtering processes already built in?
6 years ago #130267
the SQ have all Steps.
But I want too find the way of filtering.
At the moment the filtersteps are unknown.
The Question is:
How can we generate and filtering profitable Strategies?
At the moment it is only an trail and error.
With this tools I can find a process of filtering profitable strategies.
I am sorry it does not work for you with SQ, but I can filter profitable strategies from the generated strategies directly within SQ with ease and most other users here can to. It has all the filters you need + robustness tests, just look at the “ranking options” page and go from there.
You might also need to read the SQ manual and there are several articles where Mark builds strategies and filters them step by step at http://www.strategyquant.com
I disagree, it isn’t trial and error to find strategies, there is a process that works, see Mark’s tutorials. We have strategy generation, improvement, in sample and out of sample testing, Monte Carlo and walk forward to zoom in on the best strategies. Then there is backtesting using tick data in MetaTrader to confirm the results.
And then the final test, which is live trading with very small lot sizes.
I don’t think you need any other software, just a good process to follow and conviction to trade the best strategies found.
Yes, it doesn´t get any better for the filtering process than what SQ has already built in!
Ranking options by custom fitness.
Built into SQ.
I have a databank with only 10 strategies after 2million random gen right now all auto filtered to my design.
Want fitness based on Net profit and low DD? Check net profit and DD/DD%. Want to add in smooth equity curve? Check stability. Give it a weight for 2. Want even better equity curve? Check standard deviation. You make make nearly infinite combinations with filters + weights.
Use Portfolio (if exists) option if you test and build your strategies using multiple data. I build my strategies on 5 different data sources for only 1 pair for maximum robustness and to eliminate curve fitting.
Thats how you filter in SQ and it is awesome.
6 years ago #130282
OTS tests on multiple data sources, then OTS testing on different pairs or TF; Analyze Analyze Analyze. Then massive robustness tests, Analyze, Analyze Analyze; Then you have some average/good strategies; Analyze and use systems which have different exit methods, different trading style. Personally im using different 5 systems on each pair on 30 Min and 1H TF . And it works trust me/us .
Remember on worse then real spreads,slippages, etc . . .
@ Clonex what size RT test do you use right before finalizing a strategy for live production. I was lucky enough to talk to some authors who are professional traders and who worked with algos in the banking sector and they said 100,000 RT tests minimum was the industry standard (on that level).
Obviously not all of us have super computers. I been doing 1000 for the last year. I might up it to 5k, even 10k for final production.
6 years ago #130291
@ Treshold. 200- 500 tests…. in case of more SQ often freeze. In next generetion my strategies i have plans tu use multiple robustness test methodology.
1. make robustness test for ( almost ) all methodologies
3. do specific robustness test ( highest priority methodology)
5. repeat 3. step
Currently, I use one big robbustness test in a final stage of development. You?
What do you thing?
Similar. During retest and filtering strategies i’ll do ~500, and for final stage ~1000 but considering doing more so the test results are more stable and consistent.
6 years ago #130382
Do you found profitabel strategies on demoaccount ?
At the moment it is not clear how I can filter?
I think the most importend part is the RT. I use RT with N=500.
It is possible that my expectations are too high?
I found many strategies with a good backtest (at least 10 years, H1)
But not all strategies are good.
If I install a portfolio of 20 ‘good’ strategies only 10 are profitabel. I don´t know why.
Somtimes there are more profitable strategies, somtimes there are less profitable strategies.
Normaly I have to check the generation and filtering, I have done more times in the past but no generation and filtering process
fullfill this test.
I will write my method:
a1) Generation of strategies from 1.2004-1.2012 H1
b1) Filtering of strategies (Robustnesstest….. etc)
c1) Endtest: I will check the result with the last portfolio from 1.2012-1.2013
a2) Generation of strategies from 2.2004-2.2012 H1
b2) Filtering of strategies (Robustnesstest….. etc)
c2) Endtest2: I will check the result with the last portfolio from 2.2012-2.2013
a20) Generation of strategies from 1.2005-1.2013 H1
b20) Filtering of strategies (Robustnesstest….. etc)
c20) Endtest20: I will check the result with the last portfolio from 2.2014-2.2015
All 20 Endtests have to be positive with SQ-Backtest if the generation/filtering-process works.
But this was not the case in the past.
Test your strategies on additional broker data all for the same currency pair. If my strategy passes Duka EURUSD h1 but fails HistData EURUSD h1or has very different equity curve, I know it was curve fitted/over optimized (even if I didn’t do optimization yet) to duka data. I do this with 5 different data sources. My strategies must be robust enough to work on any broker data.
If it works on main data but not additional data for that pair it was curve fitted and over optimized to the main data (even if you didn’t optimize it, just because of random gen or genetic) and will likely fail going forward.
Your OOS testing seems pretty robust though.
If you build your strategies on 10 years of data, what do you mean some are not profitable? They have already exceeded their max historical DD? They already have exceeded their max historical stagnation? Or did they just not make money the first month or 2 so you decided they dont work? You must know their history. If they haven’t exceed their historical DD or historical stagnation, then they are probably in a regular DD or stagnation period.
If they have already exceeded their max historical DD and max historical stagnation…
They are over optimized. Build using additional data from more brokers. And build using more robust rules. ATR based stops and targets is an important 1. Are you using larger spreads than normal? 2.5+ for EURUSD? Adding 1-2 Slippage? Do they have enough trades to draw a reliable conclusion from? I like 500+ minimum for 10 years.
Viewing 13 posts - 1 through 13 (of 13 total)