# Need some help calculating standard error of multiple regression coefficients

#### joseph.ej

##### New Member
Hello. I am an undergrad student not very familiar with advanced statistics. Thus, I figured someone on this forum could help me in this regard:

The following is a webpage that calculates estimated regression coefficients for multiple linear regressions

http://people.hofstra.edu/stefan_Waner/realworld/multlinreg.html.

I would like to add on to the source code, so that I can figure out the standard error for each of the coefficients estimates in the regression. I don't understand the terminology in the source code, so I figured someone here might in order to show me how to calculate the std errors.

I would like to be able to figure this out as soon as possible. Thank you for your help.

#### Dragan

##### Super Moderator
Hello. I am an undergrad student not very familiar with advanced statistics. Thus, I figured someone on this forum could help me in this regard:

The following is a webpage that calculates estimated regression coefficients for multiple linear regressions

http://people.hofstra.edu/stefan_Waner/realworld/multlinreg.html.

I would like to add on to the source code, so that I can figure out the standard error for each of the coefficients estimates in the regression. I don't understand the terminology in the source code, so I figured someone here might in order to show me how to calculate the std errors.

I would like to be able to figure this out as soon as possible. Thank you for your help.
You might the find this useful. Someone else asked me the (exact) same question a few weeks ago.

The standard error for a regression coefficients is:

Se(bi) = Sqrt [MSE / (SSXi * TOLi) ]

where MSE is the mean squares for error from the overall ANOVA summary, SSXi is the sum of squares for the i-th independent variable, and TOLi is the tolerance associated with the i-th independent variable.

TOLi = 1 - Ri^2, where Ri^2 is determined by regressing Xi on all the other independent variables in the model.

-- Dragan

#### joseph.ej

##### New Member
Thanks, I was wondering if this can be shown algebraically or numerically with some data, without matrix algebra. Since I am not sure how the numerical output is derived with javascript.

here is some sample data.
http://www.egwald.ca/statistics/electiontable2004.php
I am not sure how it goes from the data to the estimates and then to the standard deviations. If you could show me, I would really appreciate it.

Sorry, I am not very literate in advanced stat methods.
Thanks again

#### bluesmoke

##### New Member
Formula for calculating the standard error of the intercept for multiple regression

Thanks for the info on calculating the standard error of the multiple regression coefficients. I was wondering what formula is used for calculating the standard error of the constant term (or intercept). Any help would be greatly appreciated.

Thanks.

#### Dragan

##### Super Moderator
Thanks for the info on calculating the standard error of the multiple regression coefficients. I was wondering what formula is used for calculating the standard error of the constant term (or intercept). Any help would be greatly appreciated.

Thanks.

In general, obtain the estimated variance-covariance matrix as (in matrix form):

S^2{b} = MSE * (X^T * X)^-1

The standard error for the intercept term, s(b0), will be the square root of the element in the first column and first row.

For a simple regression the standard error for the intercept term can be easily obtained from:

s{bo} = StdErrorReg * Sqrt [ SumX^2 / (N * SSx) ]

where StdErrorReg is the standard error of the estimate for the regression line and SSx is the sums of squares for X (the independent variable).

#### chinghm

##### New Member
Std error of intercept for multi-regression

HI

What will be the approprate equation in a multiple regression for s{bo}?

For a simple regression the standard error for the intercept term can be easily obtained from:

s{bo} = StdErrorReg * Sqrt [ SumX^2 / (N * SSx) ]

where StdErrorReg is the standard error of the estimate for the regression line and SSx is the sums of squares for X (the independent variable).[/QUOTE]

#### backkom

##### New Member
Need help

Hi,

Could you pls show me the formula to archive the S{b0} in case of multi-regression.
I need it in an emergency.

#### Dragan

##### Super Moderator
Hi,

Could you pls show me the formula to archive the S{b0} in case of multi-regression.
I need it in an emergency.

Well, it is as I said above.

I'll repeat:

In general, obtain the estimated variance-covariance matrix as (in matrix form):

S^2{b} = MSE * (X^T * X)^-1

The standard error for the intercept term, s{b0}, will be the square root of the element in the first column and first row.

I think this is clear.

#### backkom

##### New Member
Well, it is as I said above.

I'll repeat:

In general, obtain the estimated variance-covariance matrix as (in matrix form):

S^2{b} = MSE * (X^T * X)^-1

The standard error for the intercept term, s{b0}, will be the square root of the element in the first column and first row.

I think this is clear.
Hic,
sorry for any inconvenience, but i'm not a statistican, so please show me the visual formula.
Thanks alot.

#### Dragan

##### Super Moderator
Hic,
sorry for any inconvenience, but i'm not a statistican, so please show me the visual formula.
Thanks alot.

Here is some source code to follow. It's for a simple regression but the idea can be easily extended to multiple regression.

Code:
(* Let y be the y = {3, 4, 5, 7, 9, 9, 12} and x = {1, 3, 4, 6, 7, 8, 9}.
Regress y on x
and obtain the mean square for error (MSE) which is .668965517 .. *)

(* To get the standard error use an augmented matrix for X *)

xt = {{1, 1, 1, 1, 1, 1, 1}, {1., 3, 4, 6, 7, 8, 9}};
a = MatrixForm[xt]
x = Transpose[xt];
MatrixForm[x]
Print[ "Multiply X tranpose by X"]
b = xt.x;
MatrixForm[b]
Print[ "Take the Inverse of (X^T * X)"]
c = Inverse[b];
MatrixForm[c]
mse = 0.6689655172413833;
Print[ "Now multiply each element by the MSE and take then take
the Square Root of each element on the diag which is all we care about here...
I am just going to ignore the off-diag elements"]
Print[ "The standard errors are on the diag below: Intercept .7015 and for X .1160"]
u = Sqrt[mse*c];
MatrixForm[u]

Last edited:

#### backkom

##### New Member
Here is some source code to follow. It's for a simple regression but the idea can be easily extended to multiple regression.
...
Thanks so much,
So, if i have the equation y = bo + b1*X1 + b2*X2
then, X =
(1 X11 X21)
(1 X12 X22)
(1 X13 X23)
(... )

and X^T =
(1 1 1 ...)
(X11 X12 X13 ...)
(X21 X22 X23 ...)

#### Dragan

##### Super Moderator
Thanks so much,
So, if i have the equation y = bo + b1*X1 + b2*X2
then, X =
(1 X11 X21)
(1 X12 X22)
(1 X13 X23)
(... )

and X^T =
(1 1 1 ...)
(X11 X12 X13 ...)
(X21 X22 X23 ...)

Yes, you've got the idea.

#### dl7631

##### New Member
Sorry for continuing the topic. Would you please specify what Mean Squared Error MSE is meant here?
There is so much notational confusion... Do you mean:
Sum of all squared residuals (residual being Observed Y minus Regression-estimated Y) divided by (n-p)?

#### Dragan

##### Super Moderator
Sorry for continuing the topic. Would you please specify what Mean Squared Error MSE is meant here?
There is so much notational confusion... Do you mean:
Sum of all squared residuals (residual being Observed Y minus Regression-estimated Y) divided by (n-p)?
I did specify what the MSE is in my first post. And, yes, it is as you say:

MSE = SSres / df

where df = N - p

where p includes the intercept term.