Degrees of freedom

#1
Hi,

Yesterday I did some backtesting of algorithmic trading systems to study 4-rule (n = 67) versus 2-rule (n = 65) strategies. I ran five two-sample t-tests assuming unequal variances (in Excel) to compare profit/loss (PNL), PNL/max drawdown, # trades, average trade, and profit factor.

Across the five tests, degrees of freedom varied from 76 to 125. I thought df was a defined number based on sample size, but obviously I'm wrong. Why would this vary?

Thanks,
Mark
 
#3
They are related in that I compared the same two groups along the lines of five different variables.

That makes sense if df is related to pooled variance. PNLDD is a number between -1 and 7 or so whereas PNL is between +/- 5000.

Thanks!
Mark
 

obh

Active Member
#4
Hi Mark,

You may look at multiple comparisons, to think if you need to take a smaller significance level.

As you can see it is the opposite if it is pooled variance the DF calculation is simple.
But for unequal variances, it is more complex (pooled DF)
 

obh

Active Member
#6
If I were using Bonferroni correction, could I just divide 0.05 by five for five comparisons?

Mark
Hi Mark,

Correct, but the Bonferroni is not accurate better use the more accurate, The Sidak
But the difference is minor 0.010206 instead of 0.01

The problem of both calculations, that it protects against type 1 error at the expense of increasing type 2 error.
it assumes total independence which is usually not the case.

The Holm correction is a bit more balanced.
http://www.statskingdom.com/doc_anova.html#holm_calculator
 
Last edited: