Which WFM configuration to use ?
4 replies
Massimo Scapini
3 years ago #280608
I am currently in the last (I hope) stage of strategy development, where I started to use the WFM method quite extensively in order to evaluate the strategy robustness and to get the final set of parameters before testing the strategies live.
Unfortunately, I found a big variability in results depending on the settings I used. I found a lot of comments about this topic in the training and in the Forum, but honestly nothing conclusive.
So, I tried to answer in a structured way to some basic questions that probably also other SQX users still have:
1. Which is the best optimization method to use ?
2. Which is the best parametrization method to use ?
3. Which is the best precision to use ?
In order to do so, I took some tenths of EURUSD strategies that came out from my workflow (rather a standard one), cleaned the list from those that were highly correlated, and ran a WFM using a set of five different configurations:
0. Simulated IS & OOS, Suggested Parameters
1. Exact IS, Simulated OOS, Suggested Parameters
2. Exact IS, Exact OSS, Suggested Parameters
3. Simulated IS & OOS, Custom Parameters (all of them except Booleans & Not Used Exit Param.)
4. Simulated IS & OOS, Suggested Parameters, 1Min precision
The WFM config is attached. I used a combination of 60% Stability, 40% Net Profit % as Optimization criteria.
I finally compared the results using the 0 config as the baseline, using the following criteria:
1. Variability of the overall results. I.E. if the overall results is the same or not (e.g. both WFM Pass or Fail)
2. Variability of the Best Run. IE if the Number of runs and % OSS are the same or not
3. % Average and Std Deviation of Returns
4. % Average and Std Deviation of Stability
Results are summarized in the attached table screenshot (the full analysis is in the .xls)
Based on these results, my PERSONAL answer to the starting questions is the following:
1. Which is the best optimization method to use ?
If you want just to check the strategy robustness, all methods are more or less the same, but if want to use the strategy parameters for real trading, you cannot rely on Selected TF only simulation and you have to go for 1min also for WFM, even though it is extremely demanding in terms of test duration.
2. Which is the best parametrization method to use ?
This is probably your personal choice, depending on what you want to optimize. From these results it seems that if you focus on returns you should go for Custom, while if your priority is on having a smooth equity curve (as I prefer to) the suggested parameters could be ok.
3. Which is the best precision to use ?
See point 1.
I am really looking forward to receiving your comments about this topic !
0
Massimo Scapini
3 years ago #280626
An interesting update.
Since I noticed that despite the different best WFM result between Selected TF and 1Min precision simulations, most of the WFM equity curves are rather similar.
So, I simulated the last 6 Yrs of the Selected TF Only and of the 1min WFM using the best WFM result for each and compared each pair.
The correlation of the results is quite high (weekly average > 0,7), despite the different WFM outcome.
So, I would suggest:
1. If you have a powerful PC still go for 1Min precision WFM. It should take less than 1h for each run
2. If your PC is not that powerful and WFM runs take too long, you can live with the Selected TF only configuration for the WFM
0
Massimo Scapini
3 years ago #280679
Anoter intersting one.
I ran a sequential optimization on the last 6Yrs, using the same config as WFM, on 1 Min data.
The correlation with the 1 Min WFW results is outstanding!
So, even though I have a powerful PC, I will personally put in my standard Workflow as the last 2 steps:
- WFM optimization using Selected TF only in order to check the strategy robustness only on the whole testing data available
- Sequential Optimization using the last 6 Yrs with 1 min data, which is very fast, to select the running parameters for the further steps using Broker data (backtested and live)
0
Massimo Scapini
3 years ago #280680
WARNING!!!!!
FORGET Previous post. I just realized that I made a mistake selecting the wrong Databank and I sequentially optimized on top of the 1M WFM instead of the Selected Timeframe WFM, so the correlation is obvioulsly almost perfect.
Doing things right, the correlation is not bad but far for being perfect, so running a sequential optimization on top of a Selected Timeframe WFM should still to be considered a workaround for not having a powerful PC !
0
Chad Lin
1 year ago #290021
This is very helpful, thank you.
0
Viewing 4 replies - 1 through 4 (of 4 total)