Not logged in
Viewing 15 posts - 1 through 15 (of 15 total)

Forums>StrategyQuant>General Discussion>EXACT VS SIMULATED WALK FORWARD OPTIMIZATION

  • #114576|
    Customer
    473 Posts

    As you know simulated walk forward opt’ is faster than exact walk forward opt’:

     

     

    Posted 18 July 2014 – 02:41 PM

    Genetic optimization is made using genetic algorithms. 

     

    They work in a way that it creates a random population of different parameter combinations and then evolves it in subsequent generations to find the “best” set of parameters.

    Genetic optimization should be used when there are too many combinations, and where brute force method cannot be used.

     

    But it is something else than Exact and Simulated walk forward. Simulated WF is a faster mode that doesn’t run full complete walk forward analysis for every WF combination, but runs the genetic optimization only once and then “simulates” the results for different WF combinations.

     

    It is much faster but precise enough to be used normally instead of Exact method.

     

     

    Most of my strategies (90%) that pass ‘simulated WFO’ fail the ‘exact WFO’.  Has anyone else experimented with sim vs exact?  If yes, can you share your experience?  

    #134658
    Customer
    473 Posts

    Hmmm…does anyone use the EXACT walk forward optimization or am I the only one?

    #134662
    Customer
    535 Posts

    I don’t but majority of my strategies were made in EA Wizard with only shitty MT4 optimization too, but more than a few on my best are from SQ.

    Pass/fail subjective to % threshold (I use 80%), how wide your parameters are being optimizied/amount of iterations, and how many matrix runs you are doing as you know but worth noting. For OOS I do 10-20 step 5. For # of runs I’ll usually do 8-14 from 2004 step 2 (lets assume its H1 timeframe with 1000 trades).

    Mark would be the expert but I assume from the feature he made it to use himself, and the way he describes it in your quote it seems he prefers it.

    #135776
    Customer
    284 Posts

    what is the difference between optimization? 

     

    simple O tells you best paramateres for all tested history without reoptimising, but you have to find out if the values are robust

     

    walk forward O is reoptimising more often in the history? has it sense to do this test if you dont know how often to reopimise?

     

    and walk forward matrix tells you the best reoptimization period?

     

     

    is that right???  i use only WFM in genetic mode. 

    #135781
    Customer
    284 Posts

    i understand simple o, but i dont see any reason to use wfo when wfm is better?

    #135792
    Moderator
    1639 Posts

    WF optimization let’s you more easily discover curve-fitting. With simple optimization it’s really more curve-fitting. Matrix tells you how often to do that ideally. Quoting another source about WFO

     

    The purpose of walk-forward test is to determine whenever the performance of optimized trading system is the realistic or the result of curve-fitting. The performance of the system can be considered realistic if it has predicitive value and performs good on unseen (out-of-sample) market data. When the system is properly designed, the real-time trading performance should be in relation to that uncovered during optimization. If the system is going to work in real trading, it must first pass a walk-forward test. In other words, we don’t really care about in-sample results as they are (or should be) always good. What matters is out-of-sample system performance. It is the realistic estimate of how the system would work in real trading and will quickly reveal any curve-fitting issues. If out-of-sample performance is poor then you should not trade such a system.

    #135813
    Customer
    284 Posts

    when speaking about WFO but how can i be sure that divide the period into 6 parts as mentioned in the article is the right value? what if strategy pass when i divide the history into 6 parts but fail when i divide it to 8 parts? what value is the right one?

     

    isnt better to do WFM straight away? i dont see the advantge of WFO. 

    #135815
    Customer
    535 Posts

    Some people don’t like WFM because its yet another optimization.

    #141657
    Customer
    66 Posts

    I want to challenge the idea that “WFO lets you discover curve fitting”.

     

    I find a lot of good strategies that pass Monte Carlo. But fail WFM because it cannot find a 3×3 area where at least 7 out of 9 criteria are met.

     

    Is it really necessary to run WFM as an additional Robustness check? I believe it is only necessary if one NEEDS to optimize it, in which case, it is a test of robustness for the optimization, not for the strategy itself.  Am I right?

     

    Given that both WFO and WFM take way too long in Exact method, the Simulated method is pretty unreliable and useless and that the strategy does not need optimization at for the next 5 years into the future, I argue that WFM is not a necessary Robustness check.

    #141689
    Karish
    Customer
    446 Posts

    WFO/WFM in my opinion and researches are not that beneficial,

    what is beneficial is WFA where you can check on multiple OOS parts that you had not curve-fitted your strategy,

     

    i’m using only MC tests with critical settings not the default, WFA i do it manually.. as part of my work flow, thats all,

     

    i do not use WFM, although i do play with WFO sometimes but it did not improve my strategies in any way that i can remember,

     

    * NOTE: i am using ONLY random generation mode so i do trust my results to be less to non-curve fitted, though if you use genetic generation mode it will be a different story if you will use optimizations on-top of that that could lead to big over-fitting..

     

    My MC settings..: http://pasteboard.co/3GJKwEjMx.png

    #142987
    Customer
    29 Posts

    WFO/WFM in my opinion and researches are not that beneficial,

    what is beneficial is WFA where you can check on multiple OOS parts that you had not curve-fitted your strategy,

     

    i’m using only MC tests with critical settings not the default, WFA i do it manually.. as part of my work flow, thats all,

     

    i do not use WFM, although i do play with WFO sometimes but it did not improve my strategies in any way that i can remember,

     

    * NOTE: i am using ONLY random generation mode so i do trust my results to be less to non-curve fitted, though if you use genetic generation mode it will be a different story if you will use optimizations on-top of that that could lead to big over-fitting..

     

    My MC settings..: http://pasteboard.co/3GJKwEjMx.png

    Dear Karish

    Thank you for your share. I am still research the different setting between simulated and exactly of WFO.

    But I also generate strategies with only random generation mode like yours.

    A Quantitative trader lives in Taiwan build own strategy by use MT4 ,Multichart ,TradingBlox & SQ.
    Also have course with Trader Training,Programming for who really want to join this trading world.

    medium.jpg medium.jpg medium.jpg medium.jpg
     
    Youtube WebSite Blog

    #143016
    Karish
    Customer
    446 Posts

    Dear Karish

    Thank you for your share. I am still research the different setting between simulated and exactly of WFO.

    But I also generate strategies with only random generation mode like yours.

    4sure :),

    Steve i wanted to update you that i am using Genetic Generation right now as i discovered that it is much more faster and logical, i find strategies with different exit rules to the same strategy which means if it finds different random other exit rules to the same entry rules that shows and boosts the believe and confidence of mine in relation to the strategy been more robust (same entry + different exits),

     

    plus i found out that genetic generation is much more faster from what i see,

    please join our Slack chat i shared my settings file there if you interested you can PM me there i have the same nickname so i could share my strategy file with you there :)

     

    good luck!

    #143023
    Customer
    29 Posts

    4sure :),

    Steve i wanted to update you that i am using Genetic Generation right now as i discovered that it is much more faster and logical, i find strategies with different exit rules to the same strategy which means if it finds different random other exit rules to the same entry rules that shows and boosts the believe and confidence of mine in relation to the strategy been more robust (same entry + different exits),

     

    plus i found out that genetic generation is much more faster from what i see,

    please join our Slack chat i shared my settings file there if you interested you can PM me there i have the same nickname so i could share my strategy file with you there :)

     

    good luck!

    Thank you very much.

    Once I use SQ before I also use Genetic Generation. But now, I only use  random generation to build strategies then I only setting one exit rule “Exit after x bars”

    Because I want to find that if strategy has a good edge on entry it should can be profit at “exir after x bars”

    Then, when I find the strategies when I use only one exit rule. And actually strategies can be profit.

    I will do robustness testing first. After pass the robustness test I will put Remaining strategies to optimizer.

    optimize the “exit after x bars”. to find that strategies if they have good result in optimizer(strategies can profit at most x bars)

    Then I put strategies into improve strategies to find better exit rules which has an edge.

    Robustness again after I find better exit rules.Finally I use WFO for final test.

    A Quantitative trader lives in Taiwan build own strategy by use MT4 ,Multichart ,TradingBlox & SQ.
    Also have course with Trader Training,Programming for who really want to join this trading world.

    medium.jpg medium.jpg medium.jpg medium.jpg
     
    Youtube WebSite Blog

    #143081
    Karish
    Customer
    446 Posts

    good idea good luck

Viewing 15 posts - 1 through 15 (of 15 total)

You must be logged in to reply to this topic.