Cov in multiple linear regression

[hanelliot]

For multiple linear regression model Y = Xβ + e, assume X is non-random.
Assume Gauss-Markov assumptions hold. Show that variance-covariance matrix of b, the least square estimates of β is σ²(X'X)^-1.

Cov(b) = Cov[(X'X)^-1 * X'Y]
= (X'X)^-1 * X'Cov(Y)X(X'X)^-1
= ... I know the rest of the steps

My questions are:
1) notation for variance-covariance matrix of b is both Var(b) or Cov(b), correct?
2) I don't understand how I go from Cov[(X'X)^-1 * X'Y] to (X'X)^-1 * X'Cov(Y)X(X'X)^-1. Can anyone clarify? Thanks

 

[vinux]

1. right
2. Use this property
Cov(AY) = A* Cov(Y) *A'

Here A = (X'X)^-1 * X'
So A' = X*(X'X)^-1
( Since (AB)' = B'A' )

 

[hanelliot]

thank you!

 

[Dragan]

For multiple linear regression model Y = Xβ + e, assume X is non-random.
Assume Gauss-Markov assumptions hold. Show that variance-covariance matrix of b, the least square estimates of β is σ²(X'X)^-1.

Cov(b) = Cov[(X'X)^-1 * X'Y]
= (X'X)^-1 * X'Cov(Y)X(X'X)^-1
= ... I know the rest of the steps

My questions are:
1) notation for variance-covariance matrix of b is both Var(b) or Cov(b), correct?
2) I don't understand how I go from Cov[(X'X)^-1 * X'Y] to (X'X)^-1 * X'Cov(Y)X(X'X)^-1. Can anyone clarify? Thanks

Here's a sketch of what you trying to get at:

Bhat = (X`X)^-1 X`y
Substituting y=XB +e in this expression gives
Bhat = (X`X)^-1 X`(XB + e)
= (X`X)^-1 X`XB + (X`X)^-1 X`e
= B + (X`X)^1 X`e

Thus,

Bhat – B = (X`X)^-1 X`e

Now by definition:

var-cov(Bhat) = E[Bhat – B)(Bhat – B)`]
= E{[ (X`X)^-1 X`e][X`X)^-1 X`e]`}
=E[ (X`X)^-1 X`ee`X(X`X)^-1 ]
where the last step is made by the fact that (AB)`=B`A`.

As such given your assumptions:

var-cov(Bhat) = (X`X)^-1 X` E[(ee`)X(X`X)^-1
=(X`X)^-1 X Sigma^2 I X(X`X)^-1

=Sigma^2(X`X)^-1

Note: I did this quickly - but it should give you the answer you're looking for.