Not logged in
Viewing 8 posts - 1 through 8 (of 8 total)

Forums>StrategyQuant>General Discussion>IS span and curve fitting

  • #113323 |
    Customer
    48 Posts

    What do you think, probability of curve fitting bias is higher for long or short IS span?

    #128766
    Customer
    256 Posts

    I think short.

    Reason: Longer wndw means more various market conditions  (trend/panic/non-trend)

    #128767
    Customer
    48 Posts

    “Longer wndw means more various market conditions”

     

    exactly, but does it also means that algorithm must fit more various market conditions, hence… more curve fitting?

    #128798
    Mark Fric
    Administrator
    1182 Posts

    I think curve fitting is not dependent that much on IS span, you can have strategy curve fitted to very small data and also to big ones.

     

    It is more ratio of strategy degrees of freedom vs IS range.

     

    Simple strategy with little parameters will be less curve fitted to long IS than to short IS.

     

     

    Longer IS range also allows you to test your strategy on more different market conditions.

    Mark
    StrategyQuant architect

    #128810
    Customer
    48 Posts

    I agree with you guys, it feels like longer IS guarantees better immunity for unexpected market conditions and less over-fit, however I would like to think about it again more carefully.

     

    Lets say that degrees of freedom is a way for measuring over-fit.

     

    “Estimates of statistical parameters can be based upon different amounts of information or data. The number of independent pieces of information that go into the estimate of a parameter is called the degrees of freedom.” (wiki)

     

    What do you think now?

    #128814
    Mark Fric
    Administrator
    1182 Posts

    I don’t think degrees of freedom is a way to measure overfit, but the more strategy has degrees of freedom the better it can be fitted to the data.

     

    So the goal is to look for strategies with lowest possible degrees of freedom that have also smallest chance of overfitting.

    Mark
    StrategyQuant architect

    #128931
    Customer
    48 Posts

    In my opinion degrees of freedom should include number of observations (I am not sure how it is now in SQ).

     

    The longer the period (number of observations) the higher degrees of freedom. Higher degrees of freedom, higher the probability that found strategy is a random coincidence. This random coincidence in my opinion is an effect of over-fitting.

     

    Can I ask, how is it with bootstrap tests? Is number of observations included in it?

     

    Regards

    #128934
    Mark Fric
    Administrator
    1182 Posts

    you are right, degrees of freedom are related to number of observations (testing period length).

     

    I’m not sure how this is done in bootstrap, I haven’t started with it yet.

    Mark
    StrategyQuant architect

Viewing 8 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic.