- Thread starter davidbinyam
- Start date

The standard error for a regression coefficients is:

Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]

where MSE is the mean squares for error from the overall ANOVA summaryl, SSXj is the sum of squares for the j-th independent variable, and TOLj is the tolerance associated with the j-th independent variable.

TOLj = 1 - Rj^2, where Rj^2 is determined by regressing Xj on all the other independent variables in the model.

The standard error for a regression coefficients is:

Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]

where MSE is the mean squares for error from the overall ANOVA summaryl, SSXj is the sum of squares for the j-th independent variable, and TOLj is the tolerance associated with the j-th independent variable.

TOLj = 1 - Rj^2, where Rj^2 is determined by regressing Xj on all the other independent variables in the model.

Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]

where MSE is the mean squares for error from the overall ANOVA summaryl, SSXj is the sum of squares for the j-th independent variable, and TOLj is the tolerance associated with the j-th independent variable.

TOLj = 1 - Rj^2, where Rj^2 is determined by regressing Xj on all the other independent variables in the model.

I've tried following formula Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]

but got results different from Excel and SPPS ones.

I'm searching for a 6 hours - and I see only the formula like above. But it seems in case of model with no intercept there should be other formula.

Thanks in advance for attention :wave:!

I've tried following formula Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]

but got results different from Excel and SPPS ones.

I'm searching for a 6 hours - and I see only the formula like above. But it seems in case of model with no intercept there should be other formula.

Thanks in advance for attention :wave:!

In terms of the calculation of the MSE you're going to have to exclude the intercept term from the degrees of freedom i.e. in simple regression it would be N - 1 instead of N -2.

Also, the SSX term should be expressed as raw sums of squares formula i.e. Sum(X^2) rather than in deviation form (because the intercept is zero).

In terms of the calculation of the MSE you're going to

have to exclude the intercept term from the degrees of freedom i.e. in

simple regression it would be N - 1 instead of N -2.

Also, the SSX term should be expressed as raw sums of squares formula i.e.

Sum(X^2) rather than in deviation form (because the intercept is zero).

have to exclude the intercept term from the degrees of freedom i.e. in

simple regression it would be N - 1 instead of N -2.

Also, the SSX term should be expressed as raw sums of squares formula i.e.

Sum(X^2) rather than in deviation form (because the intercept is zero).

I'm calculating TOLj and Rj2 incorrectly.

E.g. Y={1,2,3}, X={2,3,5}.

I suppose TOL1=0 since I have no more indep. variables? Is this correct?

Yes. The tolerance, TOL, would be zero.

But in both cases - TOL=0 and TOL=1 I'm getting incorrect results.

For Y={1,2,3}, X={2,3,5} easily I've got

Sum(Xi^2,i=1..3) = 38

Sum(XiYi,i=1..3) = 23.

Beta=23/38=0.605263158 (exactly as SPSS 17 and Excel's ATP)

(skipped)

SSres=0.078947368

MSE=0.039473684 (exactly as SPSS 17 and Excel's ATP)

For TOL=1: SQRT(MSE / SUM_X^2/(1-0)) = 0.091970901

For TOL=0: can't be calculated

But both SPSS 17 and Excel ATP give me

Beta[1] = 0.605263158 Beta[1] Standard Error 0.032230128

Last edited:

Thanks again for prompt reply!

But in both cases - TOL=0 and TOL=1 I'm getting incorrect results.

For Y={1,2,3}, X={2,3,5} easily I've got

Sum(Xi^2,i=1..3) = 38

Sum(XiYi,i=1..3) = 23.

Beta=23/38=0.605263158 (exactly as SPSS 17 and Excel's ATP)

(skipped)

SSres=0.078947368

MSE=0.039473684 (exactly as SPSS 17 and Excel's ATP)

For TOL=1: SQRT(MSE / SUM_X^2/(1-0)) = 0.091970901

For TOL=0: can't be calculated

But both SPSS 17 and Excel ATP give me

Beta[1] = 0.605263158 Beta[1] Standard Error 0.032230128

But in both cases - TOL=0 and TOL=1 I'm getting incorrect results.

For Y={1,2,3}, X={2,3,5} easily I've got

Sum(Xi^2,i=1..3) = 38

Sum(XiYi,i=1..3) = 23.

Beta=23/38=0.605263158 (exactly as SPSS 17 and Excel's ATP)

(skipped)

SSres=0.078947368

MSE=0.039473684 (exactly as SPSS 17 and Excel's ATP)

For TOL=1: SQRT(MSE / SUM_X^2/(1-0)) = 0.091970901

For TOL=0: can't be calculated

But both SPSS 17 and Excel ATP give me

Beta[1] = 0.605263158 Beta[1] Standard Error 0.032230128

Anyway, the correct standard error (SE) is:

SE(B) = Sqrt [ MSE / Sum(X^2) ] = Sqrt [0.039473684 / 38 ] = 0.032230128

And, that should do it

Could you kindly suggest me formula for R2ij? I'm writing the code for my thesis and I'm wondering if there is some simpler formula, that doesn't require to make a lot of calculations. Because to get Rj^2 for each indep. variable I need to regress Xj on all the other independent variables in the model, i.e. run regression procedure N -1 times, where N is independent variables count.

Thanks in advance for reply!

Could you kindly suggest me formula for R2ij? I'm writing the code for my thesis and I'm wondering if there is some simpler formula, that doesn't require to make a lot of calculations. Because to get Rj^2 for each indep. variable I need to regress Xj on all the other independent variables in the model, i.e. run regression procedure N -1 times, where N is independent variables count.

Thanks in advance for reply!

Is your goal just to compute the standard errors for each independent variable?...or is there something else beyond that?

Just standard errors for each independent variable (and in result getting CI, that's just Tvalue(95%) * SEith) :yup:

Okay, what you should do is take a more computationally efficient (Matrix Algebra) approach to do this.

Read through this link below. I provided an example on how to do this.

http://talkstats.com/showthread.php?t=5056

Okay, what you should do is take a more computationally efficient (Matrix Algebra) approach to do this.

Read through this link below. I provided an example on how to do this.

http://talkstats.com/showthread.php?t=5056

Read through this link below. I provided an example on how to do this.

http://talkstats.com/showthread.php?t=5056