+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 25

Thread: how to prove COV(y bar, estimated beta1) in linear regression equals 0

  1. #1
    Points: 2,350, Level: 29
    Level completed: 34%, Points required for next Level: 100

    Location
    Philadelphia
    Posts
    46
    Thanks
    1
    Thanked 2 Times in 1 Post

    how to prove COV(y bar, estimated beta1) in linear regression equals 0




    Yi=Xi*beta1 + beta0 + error
    Prove:
    COV(average of Yi, estimated beta1) = 0

    Thanks!
    Last edited by hehe1223; 06-28-2010 at 08:03 AM.

  2. #2
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts
    Quote Originally Posted by hehe1223 View Post
    Yi=a*beta1 + beta0 + error
    Prove:
    COV(average of Yi, estimated beta1) = 0

    Thanks!
    Is "a" just a constant?

  3. #3
    Points: 2,350, Level: 29
    Level completed: 34%, Points required for next Level: 100

    Location
    Philadelphia
    Posts
    46
    Thanks
    1
    Thanked 2 Times in 1 Post
    Quote Originally Posted by Dragan View Post
    Is "a" just a constant?
    sorry. typo. I have corrected. :P

  4. #4
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts
    Quote Originally Posted by hehe1223 View Post
    sorry. typo. I have corrected. :P
    I am sorry to tell you this, but your proposition is not correct.

    More specifically, the covariance between between the mean of Y and the estimated regression slope is not zero.

    Simply, it is:

    Cov(Ybar, b1) = Xbar*Sigma^2 / ∑Xi^2.

  5. #5
    Points: 2,350, Level: 29
    Level completed: 34%, Points required for next Level: 100

    Location
    Philadelphia
    Posts
    46
    Thanks
    1
    Thanked 2 Times in 1 Post
    Quote Originally Posted by Dragan View Post
    I am sorry to tell you this, but your proposition is not correct.

    More specifically, the covariance between between the mean of Y and the estimated regression slope is not zero.

    Simply, it is:

    Cov(Ybar, b1) = Xbar*Sigma^2 / ∑Xi^2.
    Really? Cannot agree.
    I actually figured out a troublesome way (don't think it is the expected way, thus asking here) of doing it and indeed get 0.

    Some known and provable facts:
    Var(estimated beta1) = sigma^2/Sxx
    Cov(estimated beta1, estimated beta0) = -sigma^2*xbar/Sxx (something like what you claimed for Cov(Ybar, b1))

    Cov(Ybar, b1') = E(Ybar*b1') - E(Ybar)*E(b1')
    =E(xbar*b1' + b0')*b1' -(xbar*b1+b0)*b1
    =xbar*E(b1'^2) +E(b1'*b0') -xbar*b1^2-b0*b1
    =xbar(Var(b1')+b1^2) + b1*b0 + Cov(b1', b0') - xbar*b1^2 - b0*b1
    = ...
    =0

    What do you think?

  6. #6
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts
    Quote Originally Posted by hehe1223 View Post
    Really? Cannot agree.
    I actually figured out a troublesome way (don't think it is the expected way, thus asking here) of doing it and indeed get 0.

    Some known and provable facts:
    Var(estimated beta1) = sigma^2/Sxx
    Cov(estimated beta1, estimated beta0) = -sigma^2*xbar/Sxx (something like what you claimed for Cov(Ybar, b1))

    Cov(Ybar, b1') = E(Ybar*b1') - E(Ybar)*E(b1')
    =E(xbar*b1' + b0')*b1' -(xbar*b1+b0)*b1
    =xbar*E(b1'^2) +E(b1'*b0') -xbar*b1^2-b0*b1
    =xbar(Var(b1')+b1^2) + b1*b0 + Cov(b1', b0') - xbar*b1^2 - b0*b1
    = ...
    =0

    What do you think?

    Have a look at this link. I sketched the proof in my last post.

    http://www.talkstats.com/showthread.php?t=8666

  7. #7
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts
    Quote Originally Posted by Dragan View Post
    Have a look at this link. I sketched the proof in my last post.

    http://www.talkstats.com/showthread.php?t=8666

    Okay, I think I see what is going on now.

    My original assertion (which is true) is for the special case of when the intercept term is zero. I missed that subtle point when I went back and looked at my previous post i.e. the model was specified without an intercept term.

    So, yes, in general, I believe it is correct that the covariance should be zero.

    I think a quick way to write this would be:

    Cov[\bar{Y},\tilde{\beta }_{1}]=E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-E[\tilde{\beta }_{1}])]

    =E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-\beta _{1})]

    =E[(\tilde{\beta }_{0}-\beta _{0})(\tilde{\beta }_{1}-\beta _{1})+\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}]

    =Cov[\tilde{\beta }_{0},\tilde{\beta }_{1}]+E[\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}]

    =-\bar{X}Var[\tilde{\beta }_{1}]+\bar{X}Var[\tilde{\beta }_{1}]=0

    where the first term in the last part would not appear if the intercept term is zero.

  8. #8
    Points: 2,350, Level: 29
    Level completed: 34%, Points required for next Level: 100

    Location
    Philadelphia
    Posts
    46
    Thanks
    1
    Thanked 2 Times in 1 Post
    Quote Originally Posted by Dragan View Post
    Okay, I think I see what is going on now.

    My original assertion (which is true) is for the special case of when the intercept term is zero. I missed that subtle point when I went back and looked at my previous post i.e. the model was specified without an intercept term.

    So, yes, in general, I believe it is correct that the covariance should be zero.

    I think a quick way to write this would be:

    Cov[\bar{Y},\tilde{\beta }_{1}]=E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-E[\tilde{\beta }_{1}])]

    =E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-\beta _{1})]

    =E[(\tilde{\beta }_{0}-\beta _{0})(\tilde{\beta }_{1}-\beta _{1})+\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}]

    =Cov[\tilde{\beta }_{0},\tilde{\beta }_{1}]+E[\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}]

    =-\bar{X}Var[\tilde{\beta }_{1}]+\bar{X}Var[\tilde{\beta }_{1}]=0

    where the first term in the last part would not appear if the intercept term is zero.
    Thank you so much!!
    I was wondering for a long time.
    Did not expect the intercept makes such a big difference. But it makes sense.
    Thanks again.

  9. #9
    Points: 1,383, Level: 20
    Level completed: 83%, Points required for next Level: 17

    Location
    BR
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    how to prove COV(estimated beta0, estimated beta1) in linear regression is equals??

    Yi=Xi*beta1 + beta0 + error

    Prove:
    COV(estimated beta0, estimated beta1) = 0

    Thanks!

  10. #10
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: how to prove COV(estimated beta0, estimated beta1) in linear regression is equals

    Quote Originally Posted by natswim View Post
    Yi=Xi*beta1 + beta0 + error

    Prove:
    COV(estimated beta0, estimated beta1) = 0

    Thanks!
    Where are you getting the idea from that the covariance between the estimated intercept term and the estimated slope coefficient would --in general-- be zero???

  11. #11
    Points: 1,383, Level: 20
    Level completed: 83%, Points required for next Level: 17

    Location
    BR
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: how to prove COV(estimated beta0, estimated beta1) in linear regression is equals

    Quote Originally Posted by Dragan View Post
    Where are you getting the idea from that the covariance between the estimated intercept term and the estimated slope coefficient would --in general-- be zero???

    I'm so sorry...
    I wanna know this:

    Yi=Xi*beta1 + beta0 + error

    COV(estimated beta0, estimated beta1) is ????

    Thanks!

    Sorry!!

  12. #12
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: how to prove COV(estimated beta0, estimated beta1) in linear regression is equals

    Quote Originally Posted by natswim View Post
    I'm so sorry...
    I wanna know this:

    Yi=Xi*beta1 + beta0 + error

    COV(estimated beta0, estimated beta1) is ????

    Thanks!

    Sorry!!

    You can find the proof of this in other threads here - I did it not too long ago. This question has come up a few times. It goes like this (I'll be a little neater this time):


    Cov\left [ b_{0} ,b_{1}\right ]=E\left [ \left ( b_{0}-E\left [  b_{0}\right ] \right )\left ( b_{1}-E\left [b _{1} \right ] \right ) \right ]

    =E\left [ \left ( b_{0}-\beta _{0} \right )\left (b _{1} -\beta _{1}\right ) \right ]

    =-\bar{X}E\left [ b_{1}-\beta _{1} \right ]^{2}

    =-\bar{X}Var\left [b _{1} \right ]

    =-\bar{X}\frac{\sigma ^{2}}{SS_{X}}

    where I am making use of

    b_{0}=\bar{Y}-b_{1}\bar{X}

    E\left [b _{0} \right ]=\bar{Y}-\beta _{1}\bar{X}

    giving

    \left (b _{0}-E\left [b _{0} \right ] \right )=-\bar{X}\left ( b_{1}-\beta _{1} \right ).

  13. The Following User Says Thank You to Dragan For This Useful Post:


  14. #13
    Points: 1,383, Level: 20
    Level completed: 83%, Points required for next Level: 17

    Location
    BR
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: how to prove COV(estimated beta0, estimated beta1) in linear regression is equals

    Thanks by the answer. You are brilliant!
    Last edited by natswim; 10-09-2010 at 03:48 PM.

  15. #14
    Points: 1,383, Level: 20
    Level completed: 83%, Points required for next Level: 17

    Location
    BR
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: how to prove COV(estimated beta0, estimated beta1) in linear regression is equals

    Quote Originally Posted by Dragan View Post
    You can find the proof of this in other threads here - I did it not too long ago. This question has come up a few times. It goes like this (I'll be a little neater this time):


    Cov\left [ b_{0} ,b_{1}\right ]=E\left [ \left ( b_{0}-E\left [  b_{0}\right ] \right )\left ( b_{1}-E\left [b _{1} \right ] \right ) \right ]

    =E\left [ \left ( b_{0}-\beta _{0} \right )\left (b _{1} -\beta _{1}\right ) \right ]

    =-\bar{X}E\left [ b_{1}-\beta _{1} \right ]^{2}

    =-\bar{X}Var\left [b _{1} \right ]

    =-\bar{X}\frac{\sigma ^{2}}{SS_{X}}

    where I am making use of

    b_{0}=\bar{Y}-b_{1}\bar{X}

    E\left [b _{0} \right ]=\bar{Y}-\beta _{1}\bar{X}

    giving

    \left (b _{0}-E\left [b _{0} \right ] \right )=-\bar{X}\left ( b_{1}-\beta _{1} \right ).


    Thanks by the answer. You are brilliant!

  16. #15
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: how to prove COV(estimated beta0, estimated beta1) in linear regression is equals


    Quote Originally Posted by natswim View Post
    Thanks by the answer. You are brilliant!
    Well, actually, I am not brilliant...this is fairly basic stuff.

+ Reply to Thread
Page 1 of 2 1 2 LastLast

           




Similar Threads

  1. Regression Estimated variance and Chi-square distribution
    By saaagar in forum Regression Analysis
    Replies: 0
    Last Post: 11-21-2010, 04:48 AM
  2. R^2 for linear multiple linear regression
    By raver_68 in forum Regression Analysis
    Replies: 5
    Last Post: 05-12-2010, 04:25 PM
  3. Replies: 0
    Last Post: 02-23-2009, 03:55 PM
  4. Testing if median of a sample equals zero
    By mimes in forum Statistics
    Replies: 2
    Last Post: 05-13-2008, 02:29 PM
  5. My F Value equals zero
    By sturgeon in forum Psychology Statistics
    Replies: 1
    Last Post: 09-07-2006, 12:43 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats