- Thread starter hehe1223
- Start date

I am sorry to tell you this, but your proposition is not correct.

More specifically, the covariance between between the mean of Y and the estimated regression slope is**not **zero.

Simply, it is:

Cov(Ybar, b1) = Xbar*Sigma^2 / ∑Xi^2.

More specifically, the covariance between between the mean of Y and the estimated regression slope is

Simply, it is:

Cov(Ybar, b1) = Xbar*Sigma^2 / ∑Xi^2.

I actually figured out a troublesome way (don't think it is the expected way, thus asking here) of doing it and indeed get 0.

Some known and provable facts:

Var(estimated beta1) = sigma^2/Sxx

Cov(estimated beta1, estimated beta0) = -sigma^2*xbar/Sxx (something like what you claimed for Cov(Ybar, b1))

Cov(Ybar, b1') = E(Ybar*b1') - E(Ybar)*E(b1')

=E(xbar*b1' + b0')*b1' -(xbar*b1+b0)*b1

=xbar*E(b1'^2) +E(b1'*b0') -xbar*b1^2-b0*b1

=xbar(Var(b1')+b1^2) + b1*b0 + Cov(b1', b0') - xbar*b1^2 - b0*b1

= ...

=0

What do you think?

Really? Cannot agree.

I actually figured out a troublesome way (don't think it is the expected way, thus asking here) of doing it and indeed get 0.

Some known and provable facts:

Var(estimated beta1) = sigma^2/Sxx

Cov(estimated beta1, estimated beta0) = -sigma^2*xbar/Sxx (something like what you claimed for Cov(Ybar, b1))

Cov(Ybar, b1') = E(Ybar*b1') - E(Ybar)*E(b1')

=E(xbar*b1' + b0')*b1' -(xbar*b1+b0)*b1

=xbar*E(b1'^2) +E(b1'*b0') -xbar*b1^2-b0*b1

=xbar(Var(b1')+b1^2) + b1*b0 + Cov(b1', b0') - xbar*b1^2 - b0*b1

= ...

=0

What do you think?

I actually figured out a troublesome way (don't think it is the expected way, thus asking here) of doing it and indeed get 0.

Some known and provable facts:

Var(estimated beta1) = sigma^2/Sxx

Cov(estimated beta1, estimated beta0) = -sigma^2*xbar/Sxx (something like what you claimed for Cov(Ybar, b1))

Cov(Ybar, b1') = E(Ybar*b1') - E(Ybar)*E(b1')

=E(xbar*b1' + b0')*b1' -(xbar*b1+b0)*b1

=xbar*E(b1'^2) +E(b1'*b0') -xbar*b1^2-b0*b1

=xbar(Var(b1')+b1^2) + b1*b0 + Cov(b1', b0') - xbar*b1^2 - b0*b1

= ...

=0

What do you think?

Have a look at this link. I sketched the proof in my last post.

http://www.talkstats.com/showthread.php?t=8666

Have a look at this link. I sketched the proof in my last post.

http://www.talkstats.com/showthread.php?t=8666

http://www.talkstats.com/showthread.php?t=8666

Okay, I think I see what is going on now.

My original assertion (which is true) is for the special case of when the intercept term is zero. I missed that subtle point when I went back and looked at my previous post i.e. the model was specified without an intercept term.

So, yes, in general, I believe it is correct that the covariance should be zero.

I think a quick way to write this would be:

\(Cov[\bar{Y},\tilde{\beta }_{1}]=E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-E[\tilde{\beta }_{1}])] \)

\(=E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-\beta _{1})] \)

\(=E[(\tilde{\beta }_{0}-\beta _{0})(\tilde{\beta }_{1}-\beta _{1})+\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}] \)

\(=Cov[\tilde{\beta }_{0},\tilde{\beta }_{1}]+E[\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}] \)

\(=-\bar{X}Var[\tilde{\beta }_{1}]+\bar{X}Var[\tilde{\beta }_{1}]=0 \)

where the first term in the last part would not appear if the intercept term is zero.

Okay, I think I see what is going on now.

My original assertion (which is true) is for the special case of when the intercept term is zero. I missed that subtle point when I went back and looked at my previous post i.e. the model was specified without an intercept term.

So, yes, in general, I believe it is correct that the covariance should be zero.

I think a quick way to write this would be:

\(Cov[\bar{Y},\tilde{\beta }_{1}]=E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-E[\tilde{\beta }_{1}])] \)

\(=E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-\beta _{1})] \)

\(=E[(\tilde{\beta }_{0}-\beta _{0})(\tilde{\beta }_{1}-\beta _{1})+\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}] \)

\(=Cov[\tilde{\beta }_{0},\tilde{\beta }_{1}]+E[\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}] \)

\(=-\bar{X}Var[\tilde{\beta }_{1}]+\bar{X}Var[\tilde{\beta }_{1}]=0 \)

where the first term in the last part would not appear if the intercept term is zero.

My original assertion (which is true) is for the special case of when the intercept term is zero. I missed that subtle point when I went back and looked at my previous post i.e. the model was specified without an intercept term.

So, yes, in general, I believe it is correct that the covariance should be zero.

I think a quick way to write this would be:

\(Cov[\bar{Y},\tilde{\beta }_{1}]=E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-E[\tilde{\beta }_{1}])] \)

\(=E[(\bar{Y}-E[\bar{Y}])(\tilde{\beta }_{1}-\beta _{1})] \)

\(=E[(\tilde{\beta }_{0}-\beta _{0})(\tilde{\beta }_{1}-\beta _{1})+\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}] \)

\(=Cov[\tilde{\beta }_{0},\tilde{\beta }_{1}]+E[\bar{X}(\tilde{\beta }_{1}-\beta _{1})^{2}] \)

\(=-\bar{X}Var[\tilde{\beta }_{1}]+\bar{X}Var[\tilde{\beta }_{1}]=0 \)

where the first term in the last part would not appear if the intercept term is zero.

I was wondering for a long time.

Did not expect the intercept makes such a big difference. But it makes sense.

Thanks again.

Yi=Xi*beta1 + beta0 + error

Prove:

COV(estimated beta0, estimated beta1) = 0

Thanks!:wave:

Prove:

COV(estimated beta0, estimated beta1) = 0

Thanks!:wave:

Where are you getting the idea from that the covariance between the estimated intercept term and the estimated slope coefficient would --in general-- be zero???

I'm so sorry...

I wanna know this:

Yi=Xi*beta1 + beta0 + error

COV(estimated beta0, estimated beta1) is ????

Thanks!

Sorry!!

I'm so sorry...

I wanna know this:

Yi=Xi*beta1 + beta0 + error

COV(estimated beta0, estimated beta1) is ????

Thanks!

Sorry!!

I wanna know this:

Yi=Xi*beta1 + beta0 + error

COV(estimated beta0, estimated beta1) is ????

Thanks!

Sorry!!

You can find the proof of this in other threads here - I did it not too long ago. This question has come up a few times. It goes like this (I'll be a little neater this time):

\(Cov\left [ b_{0} ,b_{1}\right ]=E\left [ \left ( b_{0}-E\left [ b_{0}\right ] \right )\left ( b_{1}-E\left [b _{1} \right ] \right ) \right ] \)

\(=E\left [ \left ( b_{0}-\beta _{0} \right )\left (b _{1} -\beta _{1}\right ) \right ] \)

\(=-\bar{X}E\left [ b_{1}-\beta _{1} \right ]^{2}\)

\(=-\bar{X}Var\left [b _{1} \right ] \)

\(=-\bar{X}\frac{\sigma ^{2}}{SS_{X}}\)

where I am making use of

\(b_{0}=\bar{Y}-b_{1}\bar{X}\)

\(E\left [b _{0} \right ]=\bar{Y}-\beta _{1}\bar{X}\)

giving

\(\left (b _{0}-E\left [b _{0} \right ] \right )=-\bar{X}\left ( b_{1}-\beta _{1} \right )\).

You can find the proof of this in other threads here - I did it not too long ago. This question has come up a few times. It goes like this (I'll be a little neater this time):

\(Cov\left [ b_{0} ,b_{1}\right ]=E\left [ \left ( b_{0}-E\left [ b_{0}\right ] \right )\left ( b_{1}-E\left [b _{1} \right ] \right ) \right ] \)

\(=E\left [ \left ( b_{0}-\beta _{0} \right )\left (b _{1} -\beta _{1}\right ) \right ] \)

\(=-\bar{X}E\left [ b_{1}-\beta _{1} \right ]^{2}\)

\(=-\bar{X}Var\left [b _{1} \right ] \)

\(=-\bar{X}\frac{\sigma ^{2}}{SS_{X}}\)

where I am making use of

\(b_{0}=\bar{Y}-b_{1}\bar{X}\)

\(E\left [b _{0} \right ]=\bar{Y}-\beta _{1}\bar{X}\)

giving

\(\left (b _{0}-E\left [b _{0} \right ] \right )=-\bar{X}\left ( b_{1}-\beta _{1} \right )\).

\(Cov\left [ b_{0} ,b_{1}\right ]=E\left [ \left ( b_{0}-E\left [ b_{0}\right ] \right )\left ( b_{1}-E\left [b _{1} \right ] \right ) \right ] \)

\(=E\left [ \left ( b_{0}-\beta _{0} \right )\left (b _{1} -\beta _{1}\right ) \right ] \)

\(=-\bar{X}E\left [ b_{1}-\beta _{1} \right ]^{2}\)

\(=-\bar{X}Var\left [b _{1} \right ] \)

\(=-\bar{X}\frac{\sigma ^{2}}{SS_{X}}\)

where I am making use of

\(b_{0}=\bar{Y}-b_{1}\bar{X}\)

\(E\left [b _{0} \right ]=\bar{Y}-\beta _{1}\bar{X}\)

giving

\(\left (b _{0}-E\left [b _{0} \right ] \right )=-\bar{X}\left ( b_{1}-\beta _{1} \right )\).

Thanks by the answer. You are brilliant!:tup:

\(Cov\left [ b_{0} ,b_{1}\right ]=E\left [ \left ( b_{0}-E\left [ b_{0}\right ] \right )\left ( b_{1}-E\left [b _{1} \right ] \right ) \right ] \)

\(=E\left [ \left ( b_{0}-\beta _{0} \right )\left (b _{1} -\beta _{1}\right ) \right ] \)

\(=-\bar{X}E\left [ b_{1}-\beta _{1} \right ]^{2}\)

\(=-\bar{X}Var\left [b _{1} \right ] \)

\(=-\bar{X}\frac{\sigma ^{2}}{SS_{X}}\)

where I am making use of

\(b_{0}=\bar{Y}-b_{1}\bar{X}\)

\(E\left [b _{0} \right ]=\bar{Y}-\beta _{1}\bar{X}\)

giving

\(\left (b _{0}-E\left [b _{0} \right ] \right )=-\bar{X}\left ( b_{1}-\beta _{1} \right )\).

The last part looks suspicious. Shouldn't it be E(Ybar) instead of Ybar??

Confused.

The last part looks suspicious. Shouldn't it be E(Ybar) instead of Ybar??

Confused.

Confused.

More specifically, the sample regression function can be expressed as:

\(Y_{i}=\hat{\beta _{0}}+\hat{\beta _{1}}X_{i}+e_{i}\)

which estimates the population regression function written as:

\(Y_{i}=\beta _{0}+\beta _{1}X_{i}+u_{i}\).

As such, it is legitimate to write,

\(\bar{Y}=\beta _{0}+\beta _{1}\bar{X}+\bar{u}\).

Now, we know that the estimate of the intercept term is an unbiased estimate i.e.

\(E\left [\hat{\beta _{0}}\right ]=\beta _{0}\)

Therefore,

\(\beta _{0}=\bar{Y}-\beta _{1}\bar{X}\)

where the mean of the stochastic disturbance term is (by assumption) zero.

What I wrote is correct hehe1223.

More specifically, the sample regression function can be expressed as:

\(Y_{i}=\hat{\beta _{0}}+\hat{\beta _{1}}X_{i}+e_{i}\)

which estimates the population regression function written as:

\(Y_{i}=\beta _{0}+\beta _{1}X_{i}+u_{i}\).

As such, it is legitimate to write,

\(\bar{Y}=\beta _{0}+\beta _{1}\bar{X}+\bar{u}\).

Now, we know that the estimate of the intercept term is an unbiased estimate i.e.

\(E\left [\hat{\beta _{0}}\right ]=\beta _{0}\)

Therefore,

\(\beta _{0}=\bar{Y}-\beta _{1}\bar{X}\)

where the mean of the stochastic disturbance term is (by assumption) zero.

More specifically, the sample regression function can be expressed as:

\(Y_{i}=\hat{\beta _{0}}+\hat{\beta _{1}}X_{i}+e_{i}\)

which estimates the population regression function written as:

\(Y_{i}=\beta _{0}+\beta _{1}X_{i}+u_{i}\).

As such, it is legitimate to write,

\(\bar{Y}=\beta _{0}+\beta _{1}\bar{X}+\bar{u}\).

Now, we know that the estimate of the intercept term is an unbiased estimate i.e.

\(E\left [\hat{\beta _{0}}\right ]=\beta _{0}\)

Therefore,

\(\beta _{0}=\bar{Y}-\beta _{1}\bar{X}\)

where the mean of the stochastic disturbance term is (by assumption) zero.

I don't think you are correct. When there are finite number of observations. Errors do not sum up to 0!

That is, again, it is as I said:

\(Cov\left [ b_{0} ,b_{1}\right ]=E\left [ \left ( b_{0}-E\left [ b_{0}\right ] \right )\left ( b_{1}-E\left [b _{1} \right ] \right ) \right ] \)

\(=E\left [ \left ( b_{0}-\beta _{0} \right )\left (b _{1} -\beta _{1}\right ) \right ] \)

\(=-\bar{X}E\left [ b_{1}-\beta _{1} \right ]^{2}\)

\(=-\bar{X}Var\left [b _{1} \right ] \)

where I am making use of

\(b_{0}=\bar{Y}-b_{1}\bar{X}\)

\(E\left [b _{0} \right ]=\bar{Y}-\beta _{1}\bar{X}\)

giving

\(\left (b _{0}-E\left [b _{0} \right ] \right )=-\bar{X}\left ( b_{1}-\beta _{1} \right )\).

Now, if you want to provide a different algebraic proof, then please do so and I'll look at, Mkay.

The problem is that you cannot assume E(Ybar)=Ybar, error terms do not sum to 0.

I later found a way to prove Cov(ybar, slopehat) = 0

if you express slopehat as Sxy/Sxx, you get numerator in terms of sum of (xi-xbar)*yi, express ybar as sum of yi/n. Cov(sum of yi, yi) = var(yi)=variance of error, because all yi are independent. make use of sum of (xi-xbar)=0, you will get final answer as 0.