+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 16

Thread: Need some help calculating standard error of multiple regression coefficients

  1. #1
    Points: 3,521, Level: 37
    Level completed: 14%, Points required for next Level: 129

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Smile Need some help calculating standard error of multiple regression coefficients




    Hello. I am an undergrad student not very familiar with advanced statistics. Thus, I figured someone on this forum could help me in this regard:

    The following is a webpage that calculates estimated regression coefficients for multiple linear regressions

    http://people.hofstra.edu/stefan_Waner/realworld/multlinreg.html.

    I would like to add on to the source code, so that I can figure out the standard error for each of the coefficients estimates in the regression. I don't understand the terminology in the source code, so I figured someone here might in order to show me how to calculate the std errors.

    I would like to be able to figure this out as soon as possible. Thank you for your help.

  2. #2
    Super Moderator
    Points: 9,842, Level: 66
    Level completed: 48%, Points required for next Level: 208
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    1,781
    Thanks
    0
    Thanked 142 Times in 128 Posts
    Quote Originally Posted by joseph.ej View Post
    Hello. I am an undergrad student not very familiar with advanced statistics. Thus, I figured someone on this forum could help me in this regard:

    The following is a webpage that calculates estimated regression coefficients for multiple linear regressions

    http://people.hofstra.edu/stefan_Waner/realworld/multlinreg.html.

    I would like to add on to the source code, so that I can figure out the standard error for each of the coefficients estimates in the regression. I don't understand the terminology in the source code, so I figured someone here might in order to show me how to calculate the std errors.

    I would like to be able to figure this out as soon as possible. Thank you for your help.
    You might the find this useful. Someone else asked me the (exact) same question a few weeks ago.

    The standard error for a regression coefficients is:


    Se(bi) = Sqrt [MSE / (SSXi * TOLi) ]

    where MSE is the mean squares for error from the overall ANOVA summary, SSXi is the sum of squares for the i-th independent variable, and TOLi is the tolerance associated with the i-th independent variable.

    TOLi = 1 - Ri^2, where Ri^2 is determined by regressing Xi on all the other independent variables in the model.

    -- Dragan

  3. #3
    Points: 3,521, Level: 37
    Level completed: 14%, Points required for next Level: 129

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks, I was wondering if this can be shown algebraically or numerically with some data, without matrix algebra. Since I am not sure how the numerical output is derived with javascript.

    here is some sample data.
    http://www.egwald.ca/statistics/electiontable2004.php
    I am not sure how it goes from the data to the estimates and then to the standard deviations. If you could show me, I would really appreciate it.

    Sorry, I am not very literate in advanced stat methods.
    Thanks again

  4. #4
    Points: 2,950, Level: 33
    Level completed: 34%, Points required for next Level: 100

    Posts
    2
    Thanks
    0
    Thanked 1 Time in 1 Post

    Formula for calculating the standard error of the intercept for multiple regression

    Thanks for the info on calculating the standard error of the multiple regression coefficients. I was wondering what formula is used for calculating the standard error of the constant term (or intercept). Any help would be greatly appreciated.

    Thanks.

  5. The Following User Says Thank You to bluesmoke For This Useful Post:


  6. #5
    Super Moderator
    Points: 9,842, Level: 66
    Level completed: 48%, Points required for next Level: 208
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    1,781
    Thanks
    0
    Thanked 142 Times in 128 Posts
    Quote Originally Posted by bluesmoke View Post
    Thanks for the info on calculating the standard error of the multiple regression coefficients. I was wondering what formula is used for calculating the standard error of the constant term (or intercept). Any help would be greatly appreciated.

    Thanks.

    In general, obtain the estimated variance-covariance matrix as (in matrix form):

    S^2{b} = MSE * (X^T * X)^-1

    The standard error for the intercept term, s(b0), will be the square root of the element in the first column and first row.

    For a simple regression the standard error for the intercept term can be easily obtained from:

    s{bo} = StdErrorReg * Sqrt [ SumX^2 / (N * SSx) ]

    where StdErrorReg is the standard error of the estimate for the regression line and SSx is the sums of squares for X (the independent variable).

  7. #6
    Points: 2,950, Level: 33
    Level completed: 34%, Points required for next Level: 100

    Posts
    2
    Thanks
    0
    Thanked 1 Time in 1 Post
    Thanks a lot for the help!

  8. #7
    Points: 2,689, Level: 31
    Level completed: 60%, Points required for next Level: 61

    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Std error of intercept for multi-regression

    HI

    What will be the approprate equation in a multiple regression for s{bo}?


    For a simple regression the standard error for the intercept term can be easily obtained from:

    s{bo} = StdErrorReg * Sqrt [ SumX^2 / (N * SSx) ]

    where StdErrorReg is the standard error of the estimate for the regression line and SSx is the sums of squares for X (the independent variable).[/QUOTE]

  9. #8
    Points: 2,445, Level: 29
    Level completed: 97%, Points required for next Level: 5

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Need help

    Hi,

    Could you pls show me the formula to archive the S{b0} in case of multi-regression.
    I need it in an emergency.

    Thanks in advance.

  10. #9
    Super Moderator
    Points: 9,842, Level: 66
    Level completed: 48%, Points required for next Level: 208
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    1,781
    Thanks
    0
    Thanked 142 Times in 128 Posts
    Quote Originally Posted by backkom View Post
    Hi,

    Could you pls show me the formula to archive the S{b0} in case of multi-regression.
    I need it in an emergency.

    Thanks in advance.
    Well, it is as I said above.

    I'll repeat:

    In general, obtain the estimated variance-covariance matrix as (in matrix form):

    S^2{b} = MSE * (X^T * X)^-1

    The standard error for the intercept term, s{b0}, will be the square root of the element in the first column and first row.

    I think this is clear.

  11. #10
    Points: 2,445, Level: 29
    Level completed: 97%, Points required for next Level: 5

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Dragan View Post
    Well, it is as I said above.

    I'll repeat:

    In general, obtain the estimated variance-covariance matrix as (in matrix form):

    S^2{b} = MSE * (X^T * X)^-1

    The standard error for the intercept term, s{b0}, will be the square root of the element in the first column and first row.

    I think this is clear.
    Hic,
    sorry for any inconvenience, but i'm not a statistican, so please show me the visual formula.
    Thanks alot.
    Please help, I just have 1 more day.

  12. #11
    Super Moderator
    Points: 9,842, Level: 66
    Level completed: 48%, Points required for next Level: 208
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    1,781
    Thanks
    0
    Thanked 142 Times in 128 Posts
    Quote Originally Posted by backkom View Post
    Hic,
    sorry for any inconvenience, but i'm not a statistican, so please show me the visual formula.
    Thanks alot.
    Please help, I just have 1 more day.


    Here is some source code to follow. It's for a simple regression but the idea can be easily extended to multiple regression.

    Code: 
    
    (* Let y be the y = {3, 4, 5, 7, 9, 9, 12} and x = {1, 3, 4, 6, 7, 8, 9}.
                Regress y on x 
          and obtain the mean square for error (MSE) which is .668965517 .. *)
    
    (* To get the standard error use an augmented matrix for X *)
    
    xt = {{1, 1, 1, 1, 1, 1, 1}, {1., 3, 4, 6, 7, 8, 9}};
    a = MatrixForm[xt]
    x = Transpose[xt];
    MatrixForm[x]
    Print[ "Multiply X tranpose by X"]
    b = xt.x;
    MatrixForm[b]
    Print[ "Take the Inverse of (X^T * X)"]
    c = Inverse[b];
    MatrixForm[c]
    mse = 0.6689655172413833;
    Print[ "Now multiply each element by the MSE and take then take 
            the Square Root of each element on the diag which is all we care about here...
            I am just going to ignore the off-diag elements"]
    Print[ "The standard errors are on the diag below: Intercept .7015 and for X .1160"]
    u = Sqrt[mse*c];
    MatrixForm[u]
    Last edited by Dragan; 04-09-2009 at 12:07 PM.

  13. #12
    Points: 2,445, Level: 29
    Level completed: 97%, Points required for next Level: 5

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Quote Originally Posted by Dragan View Post
    Here is some source code to follow. It's for a simple regression but the idea can be easily extended to multiple regression.
    ...
    Thanks so much,
    So, if i have the equation y = bo + b1*X1 + b2*X2
    then, X =
    (1 X11 X21)
    (1 X12 X22)
    (1 X13 X23)
    (... )

    and X^T =
    (1 1 1 ...)
    (X11 X12 X13 ...)
    (X21 X22 X23 ...)

  14. #13
    Super Moderator
    Points: 9,842, Level: 66
    Level completed: 48%, Points required for next Level: 208
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    1,781
    Thanks
    0
    Thanked 142 Times in 128 Posts
    Quote Originally Posted by backkom View Post
    Thanks so much,
    So, if i have the equation y = bo + b1*X1 + b2*X2
    then, X =
    (1 X11 X21)
    (1 X12 X22)
    (1 X13 X23)
    (... )

    and X^T =
    (1 1 1 ...)
    (X11 X12 X13 ...)
    (X21 X22 X23 ...)


    Yes, you've got the idea.

  15. #14
    Points: 1,384, Level: 20
    Level completed: 84%, Points required for next Level: 16

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Need some help calculating standard error of multiple regression coefficients

    Sorry for continuing the topic. Would you please specify what Mean Squared Error MSE is meant here?
    There is so much notational confusion... Do you mean:
    Sum of all squared residuals (residual being Observed Y minus Regression-estimated Y) divided by (n-p)?

  16. #15
    Super Moderator
    Points: 9,842, Level: 66
    Level completed: 48%, Points required for next Level: 208
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    1,781
    Thanks
    0
    Thanked 142 Times in 128 Posts

    Re: Need some help calculating standard error of multiple regression coefficients


    Quote Originally Posted by dl7631 View Post
    Sorry for continuing the topic. Would you please specify what Mean Squared Error MSE is meant here?
    There is so much notational confusion... Do you mean:
    Sum of all squared residuals (residual being Observed Y minus Regression-estimated Y) divided by (n-p)?
    I did specify what the MSE is in my first post. And, yes, it is as you say:

    MSE = SSres / df

    where df = N - p

    where p includes the intercept term.

+ Reply to Thread
Page 1 of 2 1 2 LastLast

           




Similar Threads

  1. Standard error of multiple regression coefficients
    By davidbinyam in forum Regression Analysis
    Replies: 14
    Last Post: 12-11-2009, 06:49 PM
  2. Standard Error of Coefficients in Multiple Regression
    By JoeA838 in forum Regression Analysis
    Replies: 1
    Last Post: 10-26-2009, 02:49 PM
  3. Calculating Standard Error (using n & pi)
    By teris in forum Statistics
    Replies: 1
    Last Post: 10-18-2009, 03:59 PM
  4. Replies: 0
    Last Post: 10-23-2008, 10:49 AM
  5. standard error of ratio of coefficients
    By dragonflyFZX in forum Statistics
    Replies: 2
    Last Post: 02-04-2007, 11:48 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats