Viewing 8 posts - 1 through 8 (of 8 total)

Forums>StrategyQuant>General Discussion>IS span and curve fitting

• #113323 |
Customer
48 Posts

What do you think, probability of curve fitting bias is higher for long or short IS span?

#128766
Customer
256 Posts

I think short.

Reason: Longer wndw means more various market conditions  (trend/panic/non-trend)

#128767
Customer
48 Posts

“Longer wndw means more various market conditions”

exactly, but does it also means that algorithm must fit more various market conditions, hence… more curve fitting?

#128798
1182 Posts

I think curve fitting is not dependent that much on IS span, you can have strategy curve fitted to very small data and also to big ones.

It is more ratio of strategy degrees of freedom vs IS range.

Simple strategy with little parameters will be less curve fitted to long IS than to short IS.

Longer IS range also allows you to test your strategy on more different market conditions.

Mark
StrategyQuant architect

#128810
Customer
48 Posts

I agree with you guys, it feels like longer IS guarantees better immunity for unexpected market conditions and less over-fit, however I would like to think about it again more carefully.

Lets say that degrees of freedom is a way for measuring over-fit.

“Estimates of statistical parameters can be based upon different amounts of information or data. The number of independent pieces of information that go into the estimate of a parameter is called the degrees of freedom.” (wiki)

What do you think now?

#128814
1182 Posts

I don’t think degrees of freedom is a way to measure overfit, but the more strategy has degrees of freedom the better it can be fitted to the data.

So the goal is to look for strategies with lowest possible degrees of freedom that have also smallest chance of overfitting.

Mark
StrategyQuant architect

#128931
Customer
48 Posts

In my opinion degrees of freedom should include number of observations (I am not sure how it is now in SQ).

The longer the period (number of observations) the higher degrees of freedom. Higher degrees of freedom, higher the probability that found strategy is a random coincidence. This random coincidence in my opinion is an effect of over-fitting.

Can I ask, how is it with bootstrap tests? Is number of observations included in it?

Regards

#128934
1182 Posts

you are right, degrees of freedom are related to number of observations (testing period length).

I’m not sure how this is done in bootstrap, I haven’t started with it yet.

Mark
StrategyQuant architect

Viewing 8 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic.