show that the sum of e(i) = 0 and the sum of e(i)*Yhat(i) = 0

[alias]

Consider a usual multiple linear regression model: Y = Xβ + ε
where Y is a (n x 1) response vector, X is a (n x p) matrix with its first column composed of 1's, β is a (p x 1) parameter vector, and ε is an (n x 1) error vector. Let b and Yhat denote the least squares estimator of β and the fitted response vector.
Let e = Y - Yhat be the residual vector.
a) Show that the sum of e(i) = 0

Note: You may use the well known results relating to the hat matrix H = [X(X'X)^-1X'] as follows:
1) H^2 = H.
2) HX = X.
3) (I - H)^2 = (I - H).
4) H is symmetric

[Dason]

Since HX = X what can you say about Hx where x is any vector in the column space of x? In this case think specifically about any column of X.

Then if we let be a the first column of X and be the identity matrix we can write:



Hopefully this can get you started and if you work with it you can get the final solution.

[alias]

Is the expected value equivalent to summation in this case?

If so I could say:
E[e] = (I-H) E(Y) = (I-H)Xb = 0. IF the operators are equivalent.

[Dason]

No it's not quite the same. But if you want you could consider X'(I - H)Y and then only look at the first row of the results (although it won't matter) if it makes it easier for you to think about. Note that X'(I - H)Y = [(I - H)'X]'Y and note that both I and H are symmetric.

[alias]

Because HX = X, and H^2 = H: HX(I-H)Y = [(HX)-(H^2X)]Y = (X-X)Y = 0.
Or would that be showing (or trying to show that summation Y*e = 0? I have to do both anyway

[Dason]

Well I think you need to be a little more careful. Remember we're trying to show that sum(e_i) = 0. So part of the question is where are you getting the sum(ei) in your formulation? I'm also not sure if distributed things properly. But the main concern is you need to link this to sum(ei). In my first post I do that. You could try to build from there.

[alias]

I can't get rid of the Y in: X'(I-H)Y

[Dason]

What would (I - H)X be equal to? I've already shown a way to essentially get you to that point (minus one small step).

[alias]

(i - h)x = xi - hx = x - x

[Dason]

Good. Hopefully you can put together all the pieces. Note that you don't need to use X'(I-H) but instead could use 1'(I - H) and you would get the result directly.

[alias]

I know I must be trying your patience but if I used 1'(I - H) wouldn't I just end up with (1 - 1)Y on the RHS?

[Dason]

... and what would be wrong with that? Think about what it is you're trying to prove again. (Although technically you'd end up with (1' - 1')Y)

[alias]

... and what would be wrong with that? (Although technically you'd end up with (1' - 1')Y)

It looks right but I thought that in post #5, HX(I-H)Y = [(HX)-(H^2X)]Y = (X-X)Y = 0, was wrong. If not I could use essentially the same thing to show that summation (Y*e) = 0.

[BGM]

I am a little bit surprised if you are not happy to see 1 - 1 lol

[alias]

I didn't mean to give that impression, 1-1 is great at 2:30am. Especially when this is due in the morning.