Expectation of variance

#1
Trying to figure out the expectation of the population variance, i.e. E[tex][S^2][/tex]getting stuck on what I'm sure will turn out to be some simple algebra.

I can get as far as:
[tex]\frac{1}{n-1}E[\displaystyle\sum ((X_i - \mu)-(\overline{X} - \mu))^2][/tex]

But my notes tell me this becomes:
[tex]\frac{1}{n-1}\displaystyle\sum E[(X_i - \mu)^2] - nE[(\overline{X} - \mu)^2][/tex]

I just don't get this step at all and gone round in circles trying to understand. Starting to lose my sanity so appreciate any help.
 

Dason

Ambassador to the humans
#3
Well there are actually quite a few steps in there so it's understandable that you don't see how they are equal right away.

Let \(A_i = X_i - \mu\) and let \(B_i = \bar{X} - \mu\). Now forget about the 1/(n-1) out front for a bit

Then what we have is
\(E [ \sum_i (A_i - B_i)^2]\)

we can take different first steps but one thing we can do immediately is recognize that the expected value operator is linear so we can bring it inside of the summation

\(= \sum_i E[(A_i - B_i)^2]\)
then expand the square
\(= \sum_i E[A_i^2 - 2A_iB_i + B_i^2]\)
once again by the linearity of the expectation operator
\(=\sum_i \left(E[A_i^2] - 2E[A_iB_i] + E[B_i^2]\right)\)
and by the linearity of summation
\(=\sum_i E[A_i^2] -2\sum_i E[A_iB_i] + \sum_i E[B_i^2]\)

Now is where it gets a little bit tricky and I actually have to get going but hopefully this is a simple start on the way to finding your answer.
 
#4
Ah, i had gone down this road, but stopped as i couldn't see how the cross-terms cancelled, or how the last term was recovered. But now i know it's the right track, i will persevere. Glad to know it's at least a little bit tricky.
 
#5
I keep wanting to say [tex]\overline{X}=\mu[/tex] to make it easy. But can we ever say this given that the population parameters are rarely actually known for certain and the sample parameters are only estimates of the population parameters?
 

BGM

TS Contributor
#6
I am quite surprised that your tutor does not demonstrate this trick to you as it appear in several places in statistical context.

Anyway I just complete the derivation started by Dason.

First, bear in mind that the sample mean \( \bar{X} \) is defined to be

\( \bar{X} = \frac {1} {n} \sum_{i=1}^n X_i \)

Then,

\( E\left[\sum_{i=1}^n ((X_i - \mu) - (\bar{X} - \mu))^2\right] \)

\( = \sum_{i=1}^n E[(X_i - \mu)^2 - 2(X_i - \mu)(\bar{X} - \mu) + (\bar{X} - \mu)^2] \)

\( = \sum_{i=1}^n E[(X_i - \mu)^2] - 2E\left[(\bar{X} - \mu) \sum_{i=1}^n (X_i - \mu)\right] + \sum_{i=1}^n E[(\bar{X} - \mu)^2] \)

\( = \sum_{i=1}^n E[(X_i - \mu)^2]
- 2E[(\bar{X} - \mu) (n\bar{X} - n\mu)] + n E[(\bar{X} - \mu)^2] \)

\( = \sum_{i=1}^n E[(X_i - \mu)^2] - 2n E[(\bar{X} - \mu)^2]
+ n E[(\bar{X} - \mu)^2] \)

\( = \sum_{i=1}^n E[(X_i - \mu)^2] - n E[(\bar{X} - \mu)^2] \)