Expected value and variance of bootstrap estimate

JennySton

New Member
Hi all!

I have a question about bootstrapping.

Let the variance estimate based on B bootstrap samples be $$\widehat{SE}^2_B = \frac{1}{B-1}\sum_{b=1}^B (\hat{\theta}^*(b) - \hat{\theta}^*(\cdot))^2$$, where B is the number of bootstrap samples, $$\hat{\theta}^*(b)$$ is the estimate based on the bth bootstrap sample and $$\hat{\theta}^*(\cdot) = \frac{1}{B}\sum_{b=1}^B \hat{\theta}^*(b)$$. Also let the ideal bootstrap variance be $$\widehat{SE}^2_\infty = SE_{\hat{F}}(\hat{\theta}^*)^2$$.

I have to show that
1) $$E_{\hat{F}}(\widehat{SE}^2_B)= \widehat{SE}^2_\infty$$
which means that the bootstrap expected value of the variance estimate that is based on B bootstrap samples is the same as the ideal bootstrap variance. (Note, that the expected value is w.r.t. $$\hat{F}$$).

2) $$E_{F}(\widehat{SE}^2_B)= E_F(\widehat{SE}^2_\infty)$$
which means that variance estimate that is based on B bootstrap samples and the ideal bootstrap variance have same expected value with respect to the true CDF F.

3) $$Var_{F}(\widehat{SE}^2_B)=> Var_F(\widehat{SE}^2_\infty)$$
which means that variance estimate which is based on B bootstrap samples has larger variance than the the ideal bootstrap variance with respect to the true CDF F.

I was able to solve the first problem using the fact that the $$\hat{\theta}^*(b)$$ are iid and $$E(\hat{\theta}^*(b)) = E(\hat{\theta}^*) \forall b$$. Then with some algebra I could show the result. However, I am not quite sure how to solve the other two problems. I was first thinking I could use the result from 1) but I think that does not work because in 1) we take the expected value with respect to $$\hat{F}$$ but in 2) and 3) with respect to F.

Jenny

Dragan

Super Moderator
Parts (2) and (3) are a bit more complicated than part (1).

If my memory is correct, I think you'll find the help you need by looking at some of Bradley Efron's earlier (1980-ish) articles.

I'd suggest that you go to JSTOR and search something like: Efron nonparametric estimates standard error.

JennySton

New Member
Thank you for the hint. I'll look and see if I can find anything.