show that the sum of e(i) = 0 and the sum of e(i)*Yhat(i) = 0

#1
Consider a usual multiple linear regression model: Y = X β + ε
where Y is a (n x 1) response vector, X is a (n x p) matrix with its first column composed of 1's, β is a (p x 1) parameter vector, and ε is an (n x 1) error vector. Let b and Yhat denote the least squares estimator of β and the fitted response vector.
Let e = Y - Yhat be the residual vector.
a) Show that the sum of e(i) = 0

Note: You may use the well known results relating to the hat matrix H = [X(X'X)^-1X'] as follows:
1) H^2 = H.
2) HX = X.
3) (I - H)^2 = (I - H).
4) H is symmetric
 

Dason

Ambassador to the humans
#2
Since HX = X what can you say about Hx where x is any vector in the column space of x? In this case think specifically about any column of X.

Then if we let \(\mathbf{1}\) be a the first column of X and \(I\) be the identity matrix we can write:

\(\sum e_i = \mathbf{1}'(Y - HY) = \mathbf{1}'(I - H)Y\)

Hopefully this can get you started and if you work with it you can get the final solution.
 
#3
Is the expected value equivalent to summation in this case?

If so I could say:
E[e] = (I-H) E(Y) = (I-H)Xb = 0. IF the operators are equivalent.
 

Dason

Ambassador to the humans
#4
No it's not quite the same. But if you want you could consider X'(I - H)Y and then only look at the first row of the results (although it won't matter) if it makes it easier for you to think about. Note that X'(I - H)Y = [(I - H)'X]'Y and note that both I and H are symmetric.
 
#5
Because HX = X, and H^2 = H: HX(I-H)Y = [(HX)-(H^2X)]Y = (X-X)Y = 0.
Or would that be showing (or trying to show that summation Y*e = 0? I have to do both anyway
 

Dason

Ambassador to the humans
#6
Well I think you need to be a little more careful. Remember we're trying to show that sum(e_i) = 0. So part of the question is where are you getting the sum(ei) in your formulation? I'm also not sure if distributed things properly. But the main concern is you need to link this to sum(ei). In my first post I do that. You could try to build from there.
 

Dason

Ambassador to the humans
#8
What would (I - H)X be equal to? I've already shown a way to essentially get you to that point (minus one small step).
 

Dason

Ambassador to the humans
#10
Good. Hopefully you can put together all the pieces. Note that you don't need to use X'(I-H) but instead could use 1'(I - H) and you would get the result directly.
 

Dason

Ambassador to the humans
#12
... and what would be wrong with that? Think about what it is you're trying to prove again. (Although technically you'd end up with (1' - 1')Y)
 
#13
... and what would be wrong with that? (Although technically you'd end up with (1' - 1')Y)
It looks right but I thought that in post #5, HX(I-H)Y = [(HX)-(H^2X)]Y = (X-X)Y = 0, was wrong. If not I could use essentially the same thing to show that summation (Y*e) = 0.
 

Dason

Ambassador to the humans
#16
It looks right but I thought that in post #5, HX(I-H)Y = [(HX)-(H^2X)]Y = (X-X)Y = 0, was wrong. If not I could use essentially the same thing to show that summation (Y*e) = 0.
You could use essentially the same thing. I didn't see why you were pluggin in HX when you just wanted 1'. I could understand X' but I didn't see the need for the H in there. It also seemed like you distributed things very sloppily. But it's the same essential idea. You should end up with blahblahblah = 0Y = 0.
 

BGM

TS Contributor
#17
Two reminders: You should follow's from Dason hints at #2. Make sure you know the dimension of the matrices and get the transpose correctly.
Also the matrix multiplication is distributive but not commutative. So you should keep the same pre/post-multiplication order after the distribution as well.