# Thread: Standard error of multiple regression coefficients

1. ## Standard error of multiple regression coefficients

I was revising my undergraduate courses of Statistics which I took before 20 years. I was exercising how to construct ANOVA table for a regression with two and three predictiors. But I couldn't obtain the standard errors of the coefficients for a test of significance becuase there is no formula in both the books of regression I bought recently. I don't want to buy a third book which again may not contain any such formula. Can anybody who knows these formulas for the standard errors leave me a message here? Thank you!!

2. Originally Posted by davidbinyam
I was revising my undergraduate courses of Statistics which I took before 20 years. I was exercising how to construct ANOVA table for a regression with two and three predictiors. But I couldn't obtain the standard errors of the coefficients for a test of significance becuase there is no formula in both the books of regression I bought recently. I don't want to buy a third book which again may not contain any such formula. Can anybody who knows these formulas for the standard errors leave me a message here? Thank you!!

The standard error for a regression coefficients is:

Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]

where MSE is the mean squares for error from the overall ANOVA summaryl, SSXj is the sum of squares for the j-th independent variable, and TOLj is the tolerance associated with the j-th independent variable.

TOLj = 1 - Rj^2, where Rj^2 is determined by regressing Xj on all the other independent variables in the model.

3. Originally Posted by Dragan
The standard error for a regression coefficients is:

Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]

where MSE is the mean squares for error from the overall ANOVA summaryl, SSXj is the sum of squares for the j-th independent variable, and TOLj is the tolerance associated with the j-th independent variable.

TOLj = 1 - Rj^2, where Rj^2 is determined by regressing Xj on all the other independent variables in the model.
Thank you! It works! I was also able to determine the coefficients and their standard error using matrix algebra. That happened to be more efficient. It is advantageous to know two ways of doing that.

4. Can anybody tell me what formula should be used for model with no intercept (i.e. zero constant)? (I know this model requires more attention and sanity checks).
I've tried following formula Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]
but got results different from Excel and SPPS ones.

I'm searching for a 6 hours - and I see only the formula like above. But it seems in case of model with no intercept there should be other formula.

Thanks in advance for attention !

5. Originally Posted by alex84
Can anybody tell me what formula should be used for model with no intercept (i.e. zero constant)? (I know this model requires more attention and sanity checks).
I've tried following formula Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]
but got results different from Excel and SPPS ones.

I'm searching for a 6 hours - and I see only the formula like above. But it seems in case of model with no intercept there should be other formula.

Thanks in advance for attention !

In terms of the calculation of the MSE you're going to have to exclude the intercept term from the degrees of freedom i.e. in simple regression it would be N - 1 instead of N -2.

Also, the SSX term should be expressed as raw sums of squares formula i.e. Sum(X^2) rather than in deviation form (because the intercept is zero).

6. Originally Posted by Dragan
In terms of the calculation of the MSE you're going to

have to exclude the intercept term from the degrees of freedom i.e. in

simple regression it would be N - 1 instead of N -2.

Also, the SSX term should be expressed as raw sums of squares formula i.e.

Sum(X^2) rather than in deviation form (because the intercept is zero).
Thanks a lot, but I still have other value It seems the problem is that

I'm calculating TOLj and Rj2 incorrectly.

E.g. Y={1,2,3}, X={2,3,5}.

I suppose TOL1=0 since I have no more indep. variables? Is this correct?

7. Originally Posted by alex84
Thanks a lot, but I still have other value It seems the problem is that

I'm calculating TOLj and Rj2 incorrectly.

E.g. Y={1,2,3}, X={2,3,5}.

I suppose TOL1=0 since I have no more indep. variables? Is this correct?

Yes. The tolerance, TOL, would be ONE.

8. Originally Posted by Dragan
Yes. The tolerance, TOL, would be zero.

But in both cases - TOL=0 and TOL=1 I'm getting incorrect results.

For Y={1,2,3}, X={2,3,5} easily I've got
Sum(Xi^2,i=1..3) = 38
Sum(XiYi,i=1..3) = 23.
Beta=23/38=0.605263158 (exactly as SPSS 17 and Excel's ATP)
(skipped)
SSres=0.078947368
MSE=0.039473684 (exactly as SPSS 17 and Excel's ATP)

For TOL=1: SQRT(MSE / SUM_X^2/(1-0)) = 0.091970901
For TOL=0: can't be calculated

But both SPSS 17 and Excel ATP give me
Beta[1] = 0.605263158 Beta[1] Standard Error 0.032230128

9. Originally Posted by alex84

But in both cases - TOL=0 and TOL=1 I'm getting incorrect results.

For Y={1,2,3}, X={2,3,5} easily I've got
Sum(Xi^2,i=1..3) = 38
Sum(XiYi,i=1..3) = 23.
Beta=23/38=0.605263158 (exactly as SPSS 17 and Excel's ATP)
(skipped)
SSres=0.078947368
MSE=0.039473684 (exactly as SPSS 17 and Excel's ATP)

For TOL=1: SQRT(MSE / SUM_X^2/(1-0)) = 0.091970901
For TOL=0: can't be calculated

But both SPSS 17 and Excel ATP give me
Beta[1] = 0.605263158 Beta[1] Standard Error 0.032230128
I’m sorry I meant to say you should ignore (or disregard) the Tolerance because you only have 1 independent variable (X) in the model.

Anyway, the correct standard error (SE) is:

SE(B) = Sqrt [ MSE / Sum(X^2) ] = Sqrt [0.039473684 / 38 ] = 0.032230128

And, that should do it

the SSX term should be expressed as raw sums of squares formula i.e. Sum(X^2) rather than in deviation form
Thanks a lot!

11. Dragan,
Could you kindly suggest me formula for R2ij? I'm writing the code for my thesis and I'm wondering if there is some simpler formula, that doesn't require to make a lot of calculations. Because to get Rj^2 for each indep. variable I need to regress Xj on all the other independent variables in the model, i.e. run regression procedure N -1 times, where N is independent variables count.

12. Originally Posted by alex84
Dragan,
Could you kindly suggest me formula for R2ij? I'm writing the code for my thesis and I'm wondering if there is some simpler formula, that doesn't require to make a lot of calculations. Because to get Rj^2 for each indep. variable I need to regress Xj on all the other independent variables in the model, i.e. run regression procedure N -1 times, where N is independent variables count.

Is your goal just to compute the standard errors for each independent variable?...or is there something else beyond that?

13. Originally Posted by Dragan
Is your goal just to compute the standard errors for each independent variable?...or is there something else beyond that?
Just standard errors for each independent variable (and in result getting CI, that's just Tvalue(95%) * SEith)

14. Originally Posted by alex84
Just standard errors for each independent variable (and in result getting CI, that's just Tvalue(95%) * SEith)

Okay, what you should do is take a more computationally efficient (Matrix Algebra) approach to do this.

Read through this link below. I provided an example on how to do this.

15. Originally Posted by Dragan
Okay, what you should do is take a more computationally efficient (Matrix Algebra) approach to do this.

Read through this link below. I provided an example on how to do this.

That works! Thank you!

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts