I have a question about bootstrapping.

Let the variance estimate based on B bootstrap samples be \(\widehat{SE}^2_B = \frac{1}{B-1}\sum_{b=1}^B (\hat{\theta}^*(b) - \hat{\theta}^*(\cdot))^2 \), where B is the number of bootstrap samples, \(\hat{\theta}^*(b)\) is the estimate based on the bth bootstrap sample and \(\hat{\theta}^*(\cdot) = \frac{1}{B}\sum_{b=1}^B \hat{\theta}^*(b)\). Also let the ideal bootstrap variance be \(\widehat{SE}^2_\infty = SE_{\hat{F}}(\hat{\theta}^*)^2\).

I have to show that

1) \(E_{\hat{F}}(\widehat{SE}^2_B)= \widehat{SE}^2_\infty\)

which means that the bootstrap expected value of the variance estimate that is based on B bootstrap samples is the same as the ideal bootstrap variance. (Note, that the expected value is w.r.t. \(\hat{F}\)).

2) \(E_{F}(\widehat{SE}^2_B)= E_F(\widehat{SE}^2_\infty)\)

which means that variance estimate that is based on B bootstrap samples and the ideal bootstrap variance have same expected value with respect to the true CDF F.

3) \(Var_{F}(\widehat{SE}^2_B)=> Var_F(\widehat{SE}^2_\infty)\)

which means that variance estimate which is based on B bootstrap samples has larger variance than the the ideal bootstrap variance with respect to the true CDF F.

I was able to solve the first problem using the fact that the \(\hat{\theta}^*(b)\) are iid and \(E(\hat{\theta}^*(b)) = E(\hat{\theta}^*) \forall b \). Then with some algebra I could show the result. However, I am not quite sure how to solve the other two problems. I was first thinking I could use the result from 1) but I think that does not work because in 1) we take the expected value with respect to \(\hat{F}\) but in 2) and 3) with respect to F.

Thanks in advance,

Jenny