+ Reply to Thread
Results 1 to 13 of 13

Thread: Importance of bias in estimators

  1. #1
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Importance of bias in estimators




    I always thought that bias was the central issue in an estimator. The one thing you had to avoid above all else and the gold standard to evaluate the value of an estimator.

    Then I read this by Greene [in an econometric text]

    "Note that no mention has been made of unbiasedness. The linear least squares estimator in the linear regression model is essentially alone in the estimators considered in this book. It is generally not possible to establish unbiasedness for any other estimator. As we saw earlier, unbiasedness is of fairly limited virtue in any event. - we found for example that the property would not differentiate an estimator baed on a sample of 10 observations from one based on 10,000. Outside the linear case, consistency is the primary requirement of an estimator. Once this is established, we consider questions of efficiency and, in most cases, whether we can rely on asmpytotic normality as the basis for statistical inference."
    Can this be true
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  2. #2
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Importance of bias in estimators

    It is saying if the normality assumption of the residuals is met, the model should provide unbiased estimate - perhaps meaning you are not making any asymptotic assumptions. There is another component at work here that I am not fully remembering but it has to do the the structure of the hat-matrix (perhaps the orthogonal projection concept) and the linear combination of its components.
    Stop cowardice, ban guns!

  3. The Following User Says Thank You to hlsmith For This Useful Post:

    noetsi (12-30-2014)

  4. #3
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Importance of bias in estimators

    Really? I read that completely different. That in fact it was impossible to know if the estimates were unbiased.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  5. #4
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Importance of bias in estimators

    Hmm, it is hard to know outside of context.
    Stop cowardice, ban guns!

  6. #5
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Importance of bias in estimators

    As is generally true there is not a lot of context in the Greene book. He jumps from element to element without detailed discussion. This is required given the amount of material he is covering, but it can be confusing [well it is to me].
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  7. #6
    TS Contributor
    Points: 22,410, Level: 93
    Level completed: 6%, Points required for next Level: 940

    Posts
    3,020
    Thanks
    12
    Thanked 565 Times in 537 Posts

    Re: Importance of bias in estimators

    You have to be careful on the term "bias" here.

    For most of the estimators, consistency is a very common and important requirement that one would like to achieve - because most of us will want the estimator to be arbitrarily close to the true parameter when the sample size increase.

    A close relationship with this is the asymptotically unbiasedness. If an estimator is asyptotically unbiased, and its variance goes to zero, then it is a consistent estimator.

    Unbiasedness is another good property. But you have to note that it is neither a necessary nor a sufficient condition for the consistency. So the text have state the importance of consistency correctly.

  8. The Following User Says Thank You to BGM For This Useful Post:

    noetsi (12-30-2014)

  9. #7
    Devorador de queso
    Points: 95,819, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,935
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: Importance of bias in estimators

    Quote Originally Posted by hlsmith View Post
    It is saying if the normality assumption of the residuals is met, the model should provide unbiased estimate - perhaps meaning you are not making any asymptotic assumptions.


    What I think it was saying was that outside of a linear model we rarely see unbiased estimators. Outside of linear models what is the 'typical' estimator? The MLE. And this isn't guaranteed to be unbiased. That's pretty much what I think it was trying to get across.
    I don't have emotions and sometimes that makes me very sad.

  10. The Following 2 Users Say Thank You to Dason For This Useful Post:

    hlsmith (12-30-2014), noetsi (12-30-2014)

  11. #8
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Importance of bias in estimators

    Agreed. I was rambling because I heard someone reference the linear combination in y-hat as unbiased, but I can't remember the details. Thanks.
    Stop cowardice, ban guns!

  12. #9
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Importance of bias in estimators

    If I understood the author, and I may not have, he feels least squares is the only estimator that you can insure or even care about unbiasedness. He talks about a wide range of estimators in his book.

    If the author is arguing that you get close to the true value with large samples (asymptotic approaches) than I guess I misunderstood what he means by bias. I understand that to mean you are not accurately estimating the population parameter which seems like the single most important thing an estimator can do. Being consistantly wrong does not seem very useful.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  13. #10
    Devorador de queso
    Points: 95,819, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,935
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: Importance of bias in estimators

    No. Least squares in linear regression is just one of the only commonly used estimators that we can easily show to be unbiased. There are other cases. For example the MLE for a poisson is unbiased.

    But your concerns are basically what the author is saying don't matter. It might seem weird to say that we don't care too much if an estimator is unbiased but to be honest consistency is something we care about more. Unbiasedness is something that isn't easy to come by regardless of what the true parameter is so we settle with being happy if our estimator in a way gets closer to the truth as your sample size increases (consistency). There are other properties that we talk about but if your estimator isn't consistent then you would need a very good reason for using it.
    I don't have emotions and sometimes that makes me very sad.

  14. #11
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Importance of bias in estimators

    Larger samples you get closer to the truth and in may cases the normal distribution or whatever distribution it is approximating.
    Stop cowardice, ban guns!

  15. #12
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Importance of bias in estimators

    What I think it was saying was that outside of a linear model we rarely see unbiased estimators. Outside of linear models what is the 'typical' estimator? The MLE. And this isn't guaranteed to be unbiased.
    I think that is the most amazing thing I learned since I came here [which covers a lot of territory obviously]. Yow. Note however, that the author talks about many estimators not just least squares and MLE.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  16. #13
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Importance of bias in estimators


    Quote Originally Posted by Dason View Post
    No. Least squares in linear regression is just one of the only commonly used estimators that we can easily show to be unbiased. There are other cases. For example the MLE for a poisson is unbiased.

    But your concerns are basically what the author is saying don't matter. It might seem weird to say that we don't care too much if an estimator is unbiased but to be honest consistency is something we care about more. Unbiasedness is something that isn't easy to come by regardless of what the true parameter is so we settle with being happy if our estimator in a way gets closer to the truth as your sample size increases (consistency). There are other properties that we talk about but if your estimator isn't consistent then you would need a very good reason for using it.
    I have not read the whole book so it is possible he does not cover the estimator you mention. I actually missed this comment by the author until this thread.

    The linear least squares estimator in the linear regression model is essentially alone in the estimators considered in this book.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats