Expected value and variance of bootstrap estimate

#1
Hi all!

I have a question about bootstrapping.

Let the variance estimate based on B bootstrap samples be \(\widehat{SE}^2_B = \frac{1}{B-1}\sum_{b=1}^B (\hat{\theta}^*(b) - \hat{\theta}^*(\cdot))^2 \), where B is the number of bootstrap samples, \(\hat{\theta}^*(b)\) is the estimate based on the bth bootstrap sample and \(\hat{\theta}^*(\cdot) = \frac{1}{B}\sum_{b=1}^B \hat{\theta}^*(b)\). Also let the ideal bootstrap variance be \(\widehat{SE}^2_\infty = SE_{\hat{F}}(\hat{\theta}^*)^2\).

I have to show that
1) \(E_{\hat{F}}(\widehat{SE}^2_B)= \widehat{SE}^2_\infty\)
which means that the bootstrap expected value of the variance estimate that is based on B bootstrap samples is the same as the ideal bootstrap variance. (Note, that the expected value is w.r.t. \(\hat{F}\)).

2) \(E_{F}(\widehat{SE}^2_B)= E_F(\widehat{SE}^2_\infty)\)
which means that variance estimate that is based on B bootstrap samples and the ideal bootstrap variance have same expected value with respect to the true CDF F.

3) \(Var_{F}(\widehat{SE}^2_B)=> Var_F(\widehat{SE}^2_\infty)\)
which means that variance estimate which is based on B bootstrap samples has larger variance than the the ideal bootstrap variance with respect to the true CDF F.

I was able to solve the first problem using the fact that the \(\hat{\theta}^*(b)\) are iid and \(E(\hat{\theta}^*(b)) = E(\hat{\theta}^*) \forall b \). Then with some algebra I could show the result. However, I am not quite sure how to solve the other two problems. I was first thinking I could use the result from 1) but I think that does not work because in 1) we take the expected value with respect to \(\hat{F}\) but in 2) and 3) with respect to F.

Thanks in advance,
Jenny
 

Dragan

Super Moderator
#2
Parts (2) and (3) are a bit more complicated than part (1).

If my memory is correct, I think you'll find the help you need by looking at some of Bradley Efron's earlier (1980-ish) articles.

I'd suggest that you go to JSTOR and search something like: Efron nonparametric estimates standard error.