Hi there, this is a bit of a back to basics question; and is one that know one really gave me a straight answer to in grad school...

It is a double barrelled question about regression:

Q1. often, depending on the program being used the first line of the out put will read "coeffient" or "Intercept" and then sometimes (in systat) there will be a further ANOVA table with "regression" in the first line. Are these the same thing? or do they each mean something different.

Q2. this is probably the most basic but most important for me to understand:

In a regression output (a recent example below); how do I interpret the intercept? In simple terms, what is it telling me? and way in this case is there high (ns p-value) - what is the null hypothesis for the intercept!?

Any help, so i can understand this (finally) would be great.

Cheers,

Phil
################################
################################
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -42.51100 37.56099 -1.132 0.267
Height 1.22714 0.04019 30.531 <2e-16 ***
##################################
###################################

Hi Phil,

Q1. often, depending on the program being used the first line of the out put will read "coeffient" or "Intercept" and then sometimes (in systat) there will be a further ANOVA table with "regression" in the first line. Are these the same thing? or do they each mean something different.

They don't mean the same thing.

In the ANOVA table, the Regression F-test is testing the null hypothesis that all B = 0, excluding the intercept. If you have only a single predictor, you would get the exact same p-value here as you would for the predictor's B in the Regression Coefficients table. Try it.

But if you have more than one predictor, it's different. The alternative hypothesis for the F test is that at least one B not equal 0. It's testing the joint effect of all the predictors. The Regression Coefficients table is giving you marginal effects of each predictor. If two or more predictors explain the same variance in Y, this effect will be reflected in the overall F-test, but not in the marginal effects t-tests.

This is why you can have an overall F test that is significant (and a decent R-squared) but none of the individual coefficients are significant.

Q2. this is probably the most basic but most important for me to understand:

In a regression output (a recent example below); how do I interpret the intercept? In simple terms, what is it telling me? and way in this case is there high (ns p-value) - what is the null hypothesis for the intercept!?

Any help, so i can understand this (finally) would be great.

Cheers,

Phil
################################
################################
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -42.51100 37.56099 -1.132 0.267
Height 1.22714 0.04019 30.531 <2e-16 ***
##################################
###################################

The intercept is the mean value of Y when all X=0.

In this example, if Height never = 0, then the intercept has no intrinsic meaning. In most regression models, there is no interest in the intercept. It doesn't tell you anything about the relationship between X and Y. You do need it to calculate predicted values, though.

But this is one reason for centering. If you rescale Height so that the mean or some other meaningful value = 0 (just subtract a constant from Height), now the intercept has a meaning. It's the mean value of Y at the chosen value of X.

Hope this helps.

Karen

The intercept is the mean value of Y when all X=0.



One clarification.

It's really the predicted value, not the mean value. If all your regression assumptions are met, these are the same (they are the same theoretically). But in practice, the actual mean of Y at X=0 will not always be the mean value predicted by the equation.

Karen

karen,

thank you very much for that. I need to think about it some more ... I might need to get back to you.

Phil

So Karen,

can you clarify...

is the non significant t-test on the intercept line in the above example telling me that the intercept is not different from zero? Or did I miss something?

Phil

Hi Phil,

Yes, that's it exactly. You haven't missed anything.

And if you're at all interested, my free training teleseminar this month is on interpreting regression coefficients. It's on 2/25 at 1pm eastern. That might be in the middle of the night for you, but I do record it. I am hoping to use some real examples that people are working on, and I still need a couple more. If you are willing, and would like some free consulting, I'd be happy to go over it with you, and appreciative. Or anyone else?

Karen