Reply

IS span and curve fitting

7 replies

matka

Customer, bbp_participant, community, 186 replies.

Visit profile

9 years ago #113323

What do you think, probability of curve fitting bias is higher for long or short IS span?

0

clonex / Ivan Hudec

Customer, bbp_participant, community, sq-ultimate, contributor, author, editor, 271 replies.

Visit profile

9 years ago #128766

I think short.

Reason: Longer wndw means more various market conditions  (trend/panic/non-trend)

0

matka

Customer, bbp_participant, community, 186 replies.

Visit profile

9 years ago #128767

“Longer wndw means more various market conditions”

 

exactly, but does it also means that algorithm must fit more various market conditions, hence… more curve fitting?

0

Mark Fric

Administrator, sq-ultimate, 2 replies.

Visit profile

9 years ago #128798

I think curve fitting is not dependent that much on IS span, you can have strategy curve fitted to very small data and also to big ones.

 

It is more ratio of strategy degrees of freedom vs IS range.

 

Simple strategy with little parameters will be less curve fitted to long IS than to short IS.

 

 

Longer IS range also allows you to test your strategy on more different market conditions.

Mark
StrategyQuant architect

0

matka

Customer, bbp_participant, community, 186 replies.

Visit profile

9 years ago #128810

I agree with you guys, it feels like longer IS guarantees better immunity for unexpected market conditions and less over-fit, however I would like to think about it again more carefully.

 

Lets say that degrees of freedom is a way for measuring over-fit.

 

“Estimates of statistical parameters can be based upon different amounts of information or data. The number of independent pieces of information that go into the estimate of a parameter is called the degrees of freedom.” (wiki)

 

What do you think now?

0

Mark Fric

Administrator, sq-ultimate, 2 replies.

Visit profile

9 years ago #128814

I don’t think degrees of freedom is a way to measure overfit, but the more strategy has degrees of freedom the better it can be fitted to the data.

 

So the goal is to look for strategies with lowest possible degrees of freedom that have also smallest chance of overfitting.

Mark
StrategyQuant architect

0

matka

Customer, bbp_participant, community, 186 replies.

Visit profile

9 years ago #128931

In my opinion degrees of freedom should include number of observations (I am not sure how it is now in SQ).

 

The longer the period (number of observations) the higher degrees of freedom. Higher degrees of freedom, higher the probability that found strategy is a random coincidence. This random coincidence in my opinion is an effect of over-fitting.

 

Can I ask, how is it with bootstrap tests? Is number of observations included in it?

 

Regards

0

Mark Fric

Administrator, sq-ultimate, 2 replies.

Visit profile

9 years ago #128934

you are right, degrees of freedom are related to number of observations (testing period length).

 

I’m not sure how this is done in bootstrap, I haven’t started with it yet.

Mark
StrategyQuant architect

0

Viewing 7 replies - 1 through 7 (of 7 total)