+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 22

Thread: Variance estimation

  1. #1
    Points: 1,589, Level: 22
    Level completed: 89%, Points required for next Level: 11

    Posts
    61
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Variance estimation




    A simple question

    we have a random sample n=2 with X, Y drawn from a normal distribution N~(0, \sigma^2). Is V=\frac{X^2}{2}+\frac{Y^2}{2} a biased estimator of \sigma^2.

    Which variance is smaller V=\frac{X^2}{2}+\frac{Y^2}{2} or S_{n-1}^{2} ?

  2. #2
    TS Contributor
    Points: 8,362, Level: 61
    Level completed: 71%, Points required for next Level: 88

    Location
    Crete, Greece
    Posts
    717
    Thanks
    0
    Thanked 35 Times in 34 Posts

    Re: Variance estimation

    this seems like a tricky question. the correct answer is S^2_n-1. But in this example V will be smaller. But this does not mean it is ubiased as you pointed out. So go with the firmula, not V, the other one. Why tricky? because it seems like V is better, but with only 2 values everything is very difficult.

  3. #3
    Points: 1,589, Level: 22
    Level completed: 89%, Points required for next Level: 11

    Posts
    61
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Variance estimation

    Thanks for your reply. Could you please elaborate more with some calculations.

    I think V is unbiased. Which of the two is smaller that I don't know. I'd check which estimator is more efficient. That's my initial thought no how the problem should be solved.

    Bias or not
    E[V]=E[\frac{X^2}{2}+\frac{Y^2}{n}]=\frac{1}{2}E[X^2]+\frac{1}{2}E[Y^2]=\frac{1}{2}E[(X-\mu)^2]+\frac{1}{2}E[(Y-\mu)^2],since \mu = 0
    =\frac{1}{2}Var(X)+\frac{1}{2}Var(Y)=Var(X_i),for general two variables of the sample size n=2

    Hence, unbiased
    Last edited by _joey; 10-31-2010 at 06:50 AM.

  4. #4
    TS Contributor
    Points: 8,362, Level: 61
    Level completed: 71%, Points required for next Level: 88

    Location
    Crete, Greece
    Posts
    717
    Thanks
    0
    Thanked 35 Times in 34 Posts

    Re: Variance estimation

    Ok I see, the variance of S^2_{n-1} is known. The variance of your estimator need to be worked out now.

    Var[V]
=V[\frac{X^2}{2}+\frac{Y^2}{2}]=\frac{1}{4}V[X^2]+\frac{1}{4}V[Y^2]=\frac{1}{2}V[X^2]
    zero covariance because they are independent. You have to find the distribution of the
    X^2 now. Transformation. Actually you need to calculate the first moment. So you need E(Z^2) where Z=X^2. I think you end up with a gamma, so again the moments are known.

  5. #5
    Devorador de queso
    Points: 95,754, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,932
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: Variance estimation

    Quote Originally Posted by Masteras View Post
    You have to find the distribution of the
    X^2 now.
    Which isn't too hard considering X is distributed as normal. If it was a standard normal then X^2 would just be a chi-square. This isn't a standard normal but you can play some tricks to help you with that.

  6. #6
    Points: 1,589, Level: 22
    Level completed: 89%, Points required for next Level: 11

    Posts
    61
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Variance estimation

    okay, thanks.
    I will try to make sense of your comments. Any further hints and suggestions will be very much appreciated.

  7. #7
    Points: 1,589, Level: 22
    Level completed: 89%, Points required for next Level: 11

    Posts
    61
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Variance estimation

    Quote Originally Posted by Masteras View Post
    But in this example V will be smaller. But this does not mean it is ubiased as you pointed out
    Is V biased or unbiased? I've worked it out to be unbiased unless my solution is incorrect.

  8. #8
    Points: 1,589, Level: 22
    Level completed: 89%, Points required for next Level: 11

    Posts
    61
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Variance estimation

    Quote Originally Posted by Masteras View Post
    Ok I see, the variance of S^2_{n-1} is known. The variance of your estimator need to be worked out now.

    Var[V]=V[\frac{X^2}{2}+\frac{Y^2}{2}]=\frac{1}{4}V[X^2]+\frac{1}{4}V[Y^2]=\frac{1}{2}V[X^2]
    zero covariance because they are independent. You have to find the distribution of the
    X^2 now. Transformation. Actually you need to calculate the first moment. So you need E(Z^2) where Z=X^2. I think you end up with a gamma, so again the moments are known.
    For two variables X,Y and n=2 S^2_{n-1} =\frac{X^2}{2}-XY +\frac{Y^2}{2}
    I will need to transform it too, I guess.

  9. #9
    TS Contributor
    Points: 22,410, Level: 93
    Level completed: 6%, Points required for next Level: 940

    Posts
    3,020
    Thanks
    12
    Thanked 565 Times in 537 Posts

    Re: Variance estimation

    I think your solution is correct.
    V is an unbiased estimator for \sigma^2
    if and only if E[V] = \sigma^2

    Var[V] in this case should the mean-squared error (MSE) of the estimator.

    One thing want to point out, is that the estimator S^2 is not useful here because
    the mean is already known here, and the estimator
    V = \frac {1} {2} [(X - 0)^2 + (Y - 0)^2] is actually directly
    using the known population mean, and also is the maximum likelihood estimator.

    Hence, the estimator
    S^2 = \left(X - \frac {X + Y} {2}\right)^2 + \left(Y - \frac {X + Y} {2}\right)^2 = \frac {1} {2} (X - Y)^2
    will give a higher variance as expected.
    (Because it incorporate the sample mean as well)

    You can compare their variance, as suggested by Dason, by considering
    the chi-square distribution. (if you are allowed to use the formula for the
    variance of the chi-square distribution directly)

    Maybe 1 more hints:
    \frac {X^2 + Y^2} {\sigma^2} \sim \chi^2(2)

    \frac {(X - Y)^2} {2\sigma^2} \sim \chi^2(1)

  10. #10
    Points: 1,589, Level: 22
    Level completed: 89%, Points required for next Level: 11

    Posts
    61
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Variance estimation

    Thanks again. I will read about chi-square distribution.

    My only concern on whether or not V is biased is this line:
    =\frac{1}{2}Var(X)+\frac{1}{2}Var(Y)=Var(X_i),for general two variables of the sample size n=2

    These two are separate random variables.

  11. #11
    Devorador de queso
    Points: 95,754, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,932
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: Variance estimation

    Quote Originally Posted by _joey View Post
    Thanks again. I will read about chi-square distribution.

    My only concern on whether or not V is biased is this line:
    =\frac{1}{2}Var(X)+\frac{1}{2}Var(Y)
=Var(X_i),for general two variables of the sample size n=2

    These two are separate random variables.
    Does it make you feel better if it's written as:

    =\frac{1}{2}Var(X)+\frac{1}{2}Var(Y) = .5\sigma^2 + .5\sigma^2 = \sigma^2
=Var(X_i)

  12. #12
    Points: 1,589, Level: 22
    Level completed: 89%, Points required for next Level: 11

    Posts
    61
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Variance estimation

    I wrote the solution and I am spending time on this topic. It's all new to me and I am not yet confident with this topic.

  13. #13
    TS Contributor
    Points: 8,362, Level: 61
    Level completed: 71%, Points required for next Level: 88

    Location
    Crete, Greece
    Posts
    717
    Thanks
    0
    Thanked 35 Times in 34 Posts

    Re: Variance estimation

    Joey thank you very much for this example first. I did it and I saw that they are the same. the variance of your estimator is the same as the unbiased one S^2_{n-1}. You gave me an idea.

  14. #14
    Points: 1,589, Level: 22
    Level completed: 89%, Points required for next Level: 11

    Posts
    61
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Variance estimation

    Quote Originally Posted by BGM View Post

    \frac {(X - Y)^2} {2\sigma^2} \sim \chi^2(1)
    I think I've solved it. I should write I used your idea to solve the problem. Var(V)=\sigma^4 and Var(s^2)=2\sigma^4 where n=2

    I've got a quick question: is there an easy way to show \frac {(X - Y)^2} {2\sigma^2} \sim \chi^2(1)?

    Err I mean for n, not just n=2.
    Last edited by _joey; 11-01-2010 at 02:28 AM.

  15. #15
    TS Contributor
    Points: 8,362, Level: 61
    Level completed: 71%, Points required for next Level: 88

    Location
    Crete, Greece
    Posts
    717
    Thanks
    0
    Thanked 35 Times in 34 Posts

    Re: Variance estimation


    for n=2 the have the same variance the V and S^2_{n-1}.
    Z=X-Y~N(0,2*sigma^2). from theory, we know it. Transformations for W=Z^2 to end up with a gamma. The U=W/a (where a=2sigma^2) to see what kind of a gamma is this (chi-square actually). I told you to take it step by step to understand. keep in mind that the variance of your V (for n=2) is 2sigma^4, the same.

+ Reply to Thread
Page 1 of 2 1 2 LastLast

           




Similar Threads

  1. variance estimation
    By avish in forum Applied Statistics
    Replies: 4
    Last Post: 01-17-2011, 03:55 AM
  2. Replies: 2
    Last Post: 12-13-2010, 11:42 PM
  3. Replies: 8
    Last Post: 05-23-2010, 02:45 PM
  4. variance estimation question
    By brakepad in forum Statistics
    Replies: 11
    Last Post: 04-13-2010, 07:27 PM
  5. Covariate Estimation & Unbiased Estimation
    By 3waytie4last in forum Statistical Research
    Replies: 0
    Last Post: 09-10-2008, 12:41 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats