I don’t really use SQ much these days; I use the SQ generated MQL4 a lot though. The message thread below contains an old workflow which may benefit some of you (who knows?); I cannot answer questions or debate simply because I’ve moved on.
September 2, 2018 4:09 pm
So I’m sending a PM.
I see that you are being very helpful in the community so I’ll be happy if I can learn a thing or two from you.. while I try to switch from manual trading to algorithm.
First things first, did you maybe write down a certain workflow that you use? I mean starting from Settings for generating strategies, MC testing etc?
Talking about development first, I am currently finding the Random Generation to be surprisingly better than genetic one, avoiding hundreds of duplicate strategies, and produces results faster. Which one do you use? Do you develop on selected timeframe only or 1 min data? Do you use OOS during development? Do you require PT during development? (I find that if I don’t require it, I get many “false” strategies, that survive due to a handful of very good trades only).
Thanks for the answer.
I start by deciding how often I want the strategy to trade.
If I want it to trade frequently (assume the 30m TF since it is the TF of your Gold model); I am going to train the model on the latest 3 month period but leave the last month for out of sample.
For lower time-frames I avoid using the simple rules, hour etc, candlesticks I also remove volume, highest lowest in range and ‘ThisBArOpen’ since it has a bug. I also move the exit rule option below stop-trailing.
I place no time restictions on when the strategy can trade at this stage since it slows strategy production.
Then I randomly generate 20,000 strategies with selected timeframe only and filter models by RETDD > 2, Wins >50%, PF >1.5, Payout ratio > 1, RT RETDD > 1 (100 runs, random, 99%), meaning I also run a robustness test while building.
After I have my 20,000 models, I run a retest on the one month out of sample and delete models with no trades, RETDD < 0.8 and wins% < 50%.
I then retest on the month prior to the 3 months training data and delete models according to 5.
I then retest on 5 months both the pre and post OOS and the training data. Delete models RETDD < 2, Payout Ratio < 1, Wins < 50%, PF < 1.5.
I then retest models on 12 months of data and delete models according to 7.
Then 24 months then 7.
Then 15 years then 7
Then 15 years 1 minute data then 7
Then 15 years real tick with spread then 7.
If I have less than 100 models I’ll run a MC 95% real tick if more then either 1min or STF depending on quantity.
I then run a parameter permutation 20% prob and 50 max change…100 runs and delete model with RETDD < 1.
Then I move onto the WF stage as described.
PT target etc: well once you start producing 100s of viable models you’re going to want a mix of everything for diversification reasons so my strategies have a bit of everything to be honest except Payout ratio < 1 or in other words the SL > PT.
Genetic: I use it as a random generator with a twist by setting the mutation and cross-over prob’ to 100, it’s interesting what it spits out.
I may also simply run it with a maximum of 10 generations.
There’s time when I am searching for the One and I’ll set the maximum generation to 1000 and force it to continue to generate for at least 500 runs…lol…again it’s interesting what it spits out however this will create lots of duplication but I’m only after one fruit so it is not a big deal.
Do not forget to view the price chart for ideas because you can create your own basic strategy then throw it in improver. Do you like MA(200) cross overs? Set the entry condition and throw it in improver, do you like to buy off the weekly Pivot? Great set a limit order Pivot buy and throw it in improver.
That should help.
September 2, 2018 4:53 pm
OH! Tick Data Suite is a must https://eareview.net/
You want to test your algos under as real as possible conditions and TDS is the only tool I know for MT4 that can do it.
September 2, 2018 4:56 pm
OH! I normally only optimise 2 parameters. Normally the limit or stop coefficients. Select strategies that fit your life style not the otherway around so it is okay to bin (or give away) a very good strategy that doesn’t match your lifestyle. Don’t fall in love with them.
September 2, 2018 4:56 pm
Thank you so much for your guidelines, I’ll give it a go in the next couple of days.
If you generate for 1H, which initial model you use? 1 year with 3 last months left for OOS?
September 2, 2018 5:05 pm
6 months/2mnths either side for 1HR, 1YR for daily but I tend to go straight to the 15 years retest for daily.
September 2, 2018 5:34 pm
Do you really believe that a retest of 15 years is necessary? I mean, if a strategy is profiting and seems robust according to all your other checks in the last 5 years, doesn’t that mean it is well adjusted to the up-to-date behaviour of the market? I always thought it’s not really relevant what happened in 2005 for example, since the way the market behaved back then was significantly different.
Thanks for clarifying
September 2, 2018 5:57 pm
1. It not necessary from a robust strategy perspective but when your strategy is in drawdown and stagnation it certainly help from the psychological perspective.
2. I used to believe exactly the same for years. I didn’t believe it was possible to build a strategy without extreme optimisation that could work for 15 years and remain profitable today and in the future. I was wrong. I believe that day to day price movement is driven by human emotions and those patterns do not change a great deal. I certainly do not want a straight equity for 15 years, I’d like to be able to look at the equity fluctuations and see changes for economic shocks such 9/11 or the great recession etc. However, I’d like the strategy to be robust enough that it can adjust to market dynamics with periodic reopts.
3. consider a small investment in the Excel addin called NUMXL for time-series analysis. There are proven methods to decide whether a time-series has move from one regime to another etc.
September 3, 2018 12:23 am
A few more questions as a follow up if you don’t mind.
1) So you don’t do different timeframe / different market tests, as the videos / ebooks suggest?
2) about “RT RETDD > 1 (100 runs, random, 99%), meaning I also run a robustness test while building.”, I didn’t quite get which robustness test is that out of the 8 available?
September 3, 2018 10:23 am
1). I perform the multi-market and timeframe test on MT4. I do not reject strategies based on this test, I use it to know how many the strategy could be deploy on. This is the stage when I might perform a few manual tweaks to the EA. If I performed these tests on SQ many excellent strategies would be deleted that otherwise would be profitable with a few tweaks.
2). I was referring to the monte carlo randomise trade order randomly at 99% confidence. I’ve found a strategy that passes this will pass (9 time out of 10) the other robustness tests with the exception of parameter permutation.
September 5, 2018 10:51 pm
Also, I’ll be happy to know your settings for WF optimisation or matrix? On how long data? Which parameters optimised? Settings?
September 6, 2018 8:05 pm
I never use the Matrix because I want my strategies to have the same optimisation days so I tend to run a WF of 30 runs with 25% OOS then I retrain according to the days from the WF. For the first WF I optimise all parameters with a parameter range of 50%. For subsequent WFs I only optimise at the most 2 parameters. The 2 parameters are normally the stop/limit order coefficient. I normally set the range between 0 to -1 or 0 to 1 depending on the trade direction. See my post in your thread for data ranges.
September 9, 2018 3:20 pm
Okay. Do you know if the random generation works really randomly or has a set sequence?
So theoretically if I run 2 instances of SQ with the same settings file & Data, will the generation sequence be exactly the same in both of them? they will go in the same order through the same sequence attempts of rules? and find same strategies in same order? Or the two runs can be totally different, even if we use same building blocks etc?
September 9, 2018 4:47 pm
- This topic was modified 2 weeks, 3 days ago by notch.
You must be logged in to reply to this topic.