+ Reply to Thread
Results 1 to 5 of 5

Thread: Testing the Normality of Errors in Regression?

  1. #1
    Points: 39, Level: 1
    Level completed: 78%, Points required for next Level: 11

    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Testing the Normality of Errors in Regression?




    Hi all,

    I have simple conceptual question:

    In the simple linear regression problem, where the true relationship is,

    y = ax + b + e

    the error terms, e, are assumed to be normally distributed N(0,\sigma^2).

    However, linear regression only yields estimates \alpha \approx a and \beta \approx b. The resulting equation is,

    y = \alpha x + \beta + \epsilon

    where \epsilon is the observable residual instead of the unobservable error e.

    1. How can one test the normality assumption of the error terms if the true errors are unobservable?

    2. The residual is an estimator of the error. What kind of estimator is it? Unbiased? Maximum likelihood?

    3. The true heart of my question is this: if the reasoning is to test the normality of the errors by testing the normality of the estimator of the errors, the residuals, how does one justify a test for normality since all of the residuals are correlated? From my understanding the residuals are all correlated by their dependence on the line of best fit. While, the tests of normality (goodness-of-fit) assume independent, identically distributed (iid) random variables.

    Thanks,
    mity

  2. #2
    Omega Contributor
    Points: 38,432, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    7,006
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Testing the Normality of Errors in Regression?

    Not sure I am getting at your questions, but the test of normality is looking for homogeneity in residuals for appropriateness of procedure.

    This is probably not the cogent remark that you want, but wouldn't many values tested for normality be bounded by some type of constraint (ceilings or floor). So in a general, they have the possibility of being correlated in the same generic way?
    Stop cowardice, ban guns!

  3. #3
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: Testing the Normality of Errors in Regression?

    Quote Originally Posted by mity View Post
    Hi all,


    3. The true heart of my question is this: if the reasoning is to test the normality of the errors by testing the normality of the estimator of the errors, the residuals, how does one justify a test for normality since all of the residuals are correlated? From my understanding the residuals are all correlated by their dependence on the line of best fit. While, the tests of normality (goodness-of-fit) assume independent, identically distributed (iid) random variables.

    Thanks,
    mity
    The major point you are missing the is that X is (classically) assumed to be fixed under repeated sampling at each level of X. Thus, the errors are iiid and normally distributed with equal variances at each level of X.....Further, there's the assumption that their is no autocorrelation between the error terms.

    In short, the structural and distributional assumptions associated with the error terms are that they are iid with expected values of zero with equal variances at each level of X under repeated sampling.

    Note that if you relax the assumption that X is random instead of fixed, then these assumption still hold.

  4. #4
    Devorador de queso
    Points: 95,995, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,938
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Testing the Normality of Errors in Regression?

    Quote Originally Posted by Dragan View Post
    In short, the structural and distributional assumptions associated with the error terms are that they are iid with expected values of zero with equal variances at each level of X under repeated sampling.
    I actually think the OP has a fairly good grasp of what is going on. Note that they said the residuals are correlated (which is true). Typically though by the time you get enough observations to do a normality test the correlation isn't strong enough to care about.
    I don't have emotions and sometimes that makes me very sad.

  5. #5
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: Testing the Normality of Errors in Regression?


    Quote Originally Posted by Dason View Post
    I actually think the OP has a fairly good grasp of what is going on.

    Okay, I'll take your thoughts in mind.

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats