Help with covariance problem for a sum

[PeterVincent]

Hi,

I would appreciate some help with showing:

Cov(S[i= 1..n] Xi, S[j= 1..m] Yj) = S[i= 1.. n]S[j= 1..m](Cov(Xi, Yj)

Where S[i= 1..n] denotes summation terms i from 1 to n; likewise for S[j= 1..m].

= Cov(X1 + X2 + ... + Xn, Y1 + Y2 + ... + Yn)

Now there might be a correlation with say X2 and Y5 so:

= Cov(X1, Y1) + Cov(X1, Y2) + ... + Cov(X1, Ym) +
Cov(X2, Y1) + Cov(X2, Y2) + ... + Cov(X2, Ym) +
.
.
. +
Cov(Xn, Y1) + Cov(Xn, Y2) + ... + Cov(Xn, Ym)

Applying the definition of Cov(X, Y) to the above I obtain the required result.

However I do not really understand why I need to sum the covariances for all the pairs as listed above. I know that if there is no correlation between an X, Y pair then the covariance will be 0 so they are not contributing to the sum. But why is the covariance of the sums equal to the sum of the covairances.

A proof would be great as it would help me understand what is really going on.

Many thanks,

Peter

[PeterVincent]

Hi,

I think that I have figured this out.

I will post my work and maybe someone will comment on it.

Peter

[PeterVincent]

I would be obliged if someone would check this:

Expextation operator is E[X]

Cov(X1 + X2, Y) = E[(X1 + X2)Y] - E[X1 + X2]E[Y] //def of covariance

= E[YX1 + YX2] - E[X1 + X2]E[Y] //distribute Y into (X1 + X2)
= E[YX1] + E[YX2] - (E[X1] + E[X2])E[Y] // property of expectation E[x + y] = E[x] + E[y]
= E[YX1] + E[YX2] - E[Y]E[X1] - E[Y]E[X2] //distribute E[Y] into (E[X1] + E[X2])

= E[YX1] - E[Y]E[X1] + E[YX2] - E[Y]E[X2] //group variables
= Cov(X1, Y) + Cov(X2, Y) //def of variance


Proved:
Cov(X1 + X2, Y) = Cov(X1, Y) + Cov(X2, Y)


Now try to extend this to general sum for X

Notation: Where S[i= 1..n] denotes summation terms i from 1 to n.
Cov(S[i= 1..n]Xi, Y) = S[i= 1..n](Cov(Xi, Yj))

= E[(S[i= 1..n]Xi) * Y] - E[(S[i= 1..n]Xi)] * E[Y] //def of covariance
= E[(X1 + X2 +...+ Xn)*Y] - E[X1 + X2 +...+ Xn]* E[Y] //expand out the sums
= E[YX1 + YX2 +...+ YXn] - E[X1 + X2 +...+ Xn]* E[Y] //distribuite the Y into (X1 + X2 +...+ Xn)
= E[YX1 + YX2 +...+ YXn] - (E[X1] + E[X2] +...+ E[Xn])* E[Y] // property of expectation E[x + y] = E[x] + E[y]
= E[YX1] + E[YX2] +...+ E[YXn] - (E[X1] + E[X2] +...+ E[Xn])* E[Y] // property of expectation E[x + y] = E[x] + E[y]
= E[YX1] + E[YX2] +...+ E[YXn] -(E[Y]E[X1] + E[Y]E[X2] + E[Y]E[Xn]) // property of expectation E[x + y] = E[x] + E[y]
= E[YX1] + E[YX2] +...+ E[YXn] - E[Y]E[X1] - E[Y]E[X2] - E[Y]E[Xn] // distribuite the -
= E[YX1] - E[Y]E[X1] + E[YX2] - E[Y]E[X2] +...+ E[YXn] - E[Y]E[Xn] // group the variables
= Cov(X1, Y) + Cov(X2, Y) +...+ Cov(Xn, Y) //def of variance
= S[i= 1..n]Cov(Xi, Y) // apply summation operator, proved

Proved:
Cov(S[i= 1..n]Xi, Y) = S[i= 1..n](Cov(Xi, Y)), call this equation *

Now try to extend for general sum of Y

Notation: Where S[j= 1..m] denotes summation terms j from 1 to m.

To prove:
Cov(S[i= 1..n]Xi, S[j= 1..m]Yj) = S[i= 1.. n]S[j= 1..m](Cov(Xi, Yj))

Start with:

Cov(S[i= 1..n]Xi, Y) = S[i= 1..n](Cov(Xi, Y))
= S[i= 1..n](Cov(Xi, S[j= 1..m]Yj)) //instantiate with Y = S[j= 1..m]Yj
= S[i= 1..n](Cov(S[j= 1..m]Yj, Xi,)) // covariance commutes
= S[i= 1..n]S[j= 1..m](Cov(Yj, Xi,)) //apply equation * with X & Y interchanged
= S[i= 1..n]S[j= 1..m](Cov(Xi, Yj)) // covariance commutes, proved

Proved:
Cov(S[i= 1..n]Xi, S[j= 1..m]Yj) = S[i= 1.. n]S[j= 1..m](Cov(Xi, Yj))

Peter

[mp83]

I think it's right (I skimmed along the lines..)

You could find this in a regression textbook in sections proving BLUEs etc. I remember I had to prove this for my Regression Analysis exams couple of years ago.