# Thread: Need some help calculating standard error of multiple regression coefficients

1. ## Need some help calculating standard error of multiple regression coefficients

Hello. I am an undergrad student not very familiar with advanced statistics. Thus, I figured someone on this forum could help me in this regard:

The following is a webpage that calculates estimated regression coefficients for multiple linear regressions

http://people.hofstra.edu/stefan_Waner/realworld/multlinreg.html.

I would like to add on to the source code, so that I can figure out the standard error for each of the coefficients estimates in the regression. I don't understand the terminology in the source code, so I figured someone here might in order to show me how to calculate the std errors.

I would like to be able to figure this out as soon as possible. Thank you for your help.

2. Originally Posted by joseph.ej
Hello. I am an undergrad student not very familiar with advanced statistics. Thus, I figured someone on this forum could help me in this regard:

The following is a webpage that calculates estimated regression coefficients for multiple linear regressions

http://people.hofstra.edu/stefan_Waner/realworld/multlinreg.html.

I would like to add on to the source code, so that I can figure out the standard error for each of the coefficients estimates in the regression. I don't understand the terminology in the source code, so I figured someone here might in order to show me how to calculate the std errors.

I would like to be able to figure this out as soon as possible. Thank you for your help.
You might the find this useful. Someone else asked me the (exact) same question a few weeks ago.

The standard error for a regression coefficients is:

Se(bi) = Sqrt [MSE / (SSXi * TOLi) ]

where MSE is the mean squares for error from the overall ANOVA summary, SSXi is the sum of squares for the i-th independent variable, and TOLi is the tolerance associated with the i-th independent variable.

TOLi = 1 - Ri^2, where Ri^2 is determined by regressing Xi on all the other independent variables in the model.

-- Dragan

3. Thanks, I was wondering if this can be shown algebraically or numerically with some data, without matrix algebra. Since I am not sure how the numerical output is derived with javascript.

here is some sample data.
http://www.egwald.ca/statistics/electiontable2004.php
I am not sure how it goes from the data to the estimates and then to the standard deviations. If you could show me, I would really appreciate it.

Sorry, I am not very literate in advanced stat methods.
Thanks again

4. ## Formula for calculating the standard error of the intercept for multiple regression

Thanks for the info on calculating the standard error of the multiple regression coefficients. I was wondering what formula is used for calculating the standard error of the constant term (or intercept). Any help would be greatly appreciated.

Thanks.

5. ## The Following User Says Thank You to bluesmoke For This Useful Post:

6. Originally Posted by bluesmoke
Thanks for the info on calculating the standard error of the multiple regression coefficients. I was wondering what formula is used for calculating the standard error of the constant term (or intercept). Any help would be greatly appreciated.

Thanks.

In general, obtain the estimated variance-covariance matrix as (in matrix form):

S^2{b} = MSE * (X^T * X)^-1

The standard error for the intercept term, s(b0), will be the square root of the element in the first column and first row.

For a simple regression the standard error for the intercept term can be easily obtained from:

s{bo} = StdErrorReg * Sqrt [ SumX^2 / (N * SSx) ]

where StdErrorReg is the standard error of the estimate for the regression line and SSx is the sums of squares for X (the independent variable).

7. Thanks a lot for the help!

8. ## Std error of intercept for multi-regression

HI

What will be the approprate equation in a multiple regression for s{bo}?

For a simple regression the standard error for the intercept term can be easily obtained from:

s{bo} = StdErrorReg * Sqrt [ SumX^2 / (N * SSx) ]

where StdErrorReg is the standard error of the estimate for the regression line and SSx is the sums of squares for X (the independent variable).[/QUOTE]

9. ## Need help

Hi,

Could you pls show me the formula to archive the S{b0} in case of multi-regression.
I need it in an emergency.

Thanks in advance.

10. Originally Posted by backkom
Hi,

Could you pls show me the formula to archive the S{b0} in case of multi-regression.
I need it in an emergency.

Thanks in advance.
Well, it is as I said above.

I'll repeat:

In general, obtain the estimated variance-covariance matrix as (in matrix form):

S^2{b} = MSE * (X^T * X)^-1

The standard error for the intercept term, s{b0}, will be the square root of the element in the first column and first row.

I think this is clear.

11. Originally Posted by Dragan
Well, it is as I said above.

I'll repeat:

In general, obtain the estimated variance-covariance matrix as (in matrix form):

S^2{b} = MSE * (X^T * X)^-1

The standard error for the intercept term, s{b0}, will be the square root of the element in the first column and first row.

I think this is clear.
Hic,
sorry for any inconvenience, but i'm not a statistican, so please show me the visual formula.
Thanks alot.
Please help, I just have 1 more day.

12. Originally Posted by backkom
Hic,
sorry for any inconvenience, but i'm not a statistican, so please show me the visual formula.
Thanks alot.
Please help, I just have 1 more day.

Here is some source code to follow. It's for a simple regression but the idea can be easily extended to multiple regression.

Code:
``````
(* Let y be the y = {3, 4, 5, 7, 9, 9, 12} and x = {1, 3, 4, 6, 7, 8, 9}.
Regress y on x
and obtain the mean square for error (MSE) which is .668965517 .. *)

(* To get the standard error use an augmented matrix for X *)

xt = {{1, 1, 1, 1, 1, 1, 1}, {1., 3, 4, 6, 7, 8, 9}};
a = MatrixForm[xt]
x = Transpose[xt];
MatrixForm[x]
Print[ "Multiply X tranpose by X"]
b = xt.x;
MatrixForm[b]
Print[ "Take the Inverse of (X^T * X)"]
c = Inverse[b];
MatrixForm[c]
mse = 0.6689655172413833;
Print[ "Now multiply each element by the MSE and take then take
the Square Root of each element on the diag which is all we care about here...
I am just going to ignore the off-diag elements"]
Print[ "The standard errors are on the diag below: Intercept .7015 and for X .1160"]
u = Sqrt[mse*c];
MatrixForm[u]``````

13. Originally Posted by Dragan
Here is some source code to follow. It's for a simple regression but the idea can be easily extended to multiple regression.
...
Thanks so much,
So, if i have the equation y = bo + b1*X1 + b2*X2
then, X =
(1 X11 X21)
(1 X12 X22)
(1 X13 X23)
(... )

and X^T =
(1 1 1 ...)
(X11 X12 X13 ...)
(X21 X22 X23 ...)

14. Originally Posted by backkom
Thanks so much,
So, if i have the equation y = bo + b1*X1 + b2*X2
then, X =
(1 X11 X21)
(1 X12 X22)
(1 X13 X23)
(... )

and X^T =
(1 1 1 ...)
(X11 X12 X13 ...)
(X21 X22 X23 ...)

Yes, you've got the idea.

15. ## Re: Need some help calculating standard error of multiple regression coefficients

Sorry for continuing the topic. Would you please specify what Mean Squared Error MSE is meant here?
There is so much notational confusion... Do you mean:
Sum of all squared residuals (residual being Observed Y minus Regression-estimated Y) divided by (n-p)?

16. ## Re: Need some help calculating standard error of multiple regression coefficients

Originally Posted by dl7631
Sorry for continuing the topic. Would you please specify what Mean Squared Error MSE is meant here?
There is so much notational confusion... Do you mean:
Sum of all squared residuals (residual being Observed Y minus Regression-estimated Y) divided by (n-p)?
I did specify what the MSE is in my first post. And, yes, it is as you say:

MSE = SSres / df

where df = N - p

where p includes the intercept term.

+ Reply to Thread
Page 1 of 2 1 2 Last

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts