+ Reply to Thread
Results 1 to 5 of 5

Thread: LinReg Assumption on errors ConfIV

  1. #1
    Points: 896, Level: 15
    Level completed: 96%, Points required for next Level: 4

    Posts
    35
    Thanks
    2
    Thanked 3 Times in 3 Posts

    LinReg Assumption on errors ConfIV




    Hi all

    why is it assumed that the errors e_i are normal distributed in linear regression, see e.g. Seber.

    What would be the influence e.g. on confidence intervals for parameter estimates, if one neglect this assumption.

  2. #2
    ggplot2orBust
    Points: 71,220, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    User with most referrers
    trinker's Avatar
    Location
    Buffalo, NY
    Posts
    4,417
    Thanks
    1,811
    Thanked 928 Times in 809 Posts

    Re: LinReg Assumption on errors ConfIV

    This is a nonstatistician's stab at your questions:
    why is it assumed that the errors e_i are normal distributed in linear regression, see e.g. Seber.
    First the word assumed means something a bit different than you may think it means. It means that in order for this test to be appropriate the data must fit this specification. So one of the assumptions of the test you are using is that data comes from a population that follows the normal distribution (notice I didn't say the data is normally distributed as this is rarely the case). If the errors are normally distributed then this is an indication that the data comes from a normally distributed population.

    What would be the influence e.g. on confidence intervals for parameter estimates, if one neglect this assumption
    According to Cohen, Cohen, Aiken and West (2003) the violation of normally distributed error terms does not lead to biased estimates. "The affect of violation of the normality assumption on significance tests and confidence intervals depends on the sample size, with problems occurring in small samples."
    "If you torture the data long enough it will eventually confess."
    -Ronald Harry Coase -

  3. The Following User Says Thank You to trinker For This Useful Post:

    Jacov (04-09-2012)

  4. #3
    Points: 896, Level: 15
    Level completed: 96%, Points required for next Level: 4

    Posts
    35
    Thanks
    2
    Thanked 3 Times in 3 Posts

    Re: LinReg Assumption on errors ConfIV

    Thx for reply,

    actually I am not interested in data but in mathematical derivations.

    In LinReg (see Seber) one "assumes" that e_i ~ N, therefore one can show
    that where p is the number of
    parameter, n the sample size, s^2 the variance estimator and \sigma^2 the true
    variance. Based on this result the ConfIV based on a t-distribution is constructed.

    Now my problem:

    What would happen if e_i is not normal distributed . Then the whole argumenation does not work anymore.

    Are there any asymptotical / approximate results available? For example based on
    a central limit theorem?

    Do you understand my issue? (It is about mathematical argumentation and not about data)

    It totally agree with your presented citation from Cohen in practice !

  5. #4
    Devorador de queso
    Points: 95,819, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,935
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: LinReg Assumption on errors ConfIV

    Yes there are CLTs that can be applied. For instance if all we assume is that the error terms satisfy E[e_i] = 0 and Var[e_i] = \sigma^2 < \infty then along with a mild condition on the predictor we can show that the parameter estimates are asymptotically normal.
    I don't have emotions and sometimes that makes me very sad.

  6. The Following User Says Thank You to Dason For This Useful Post:

    Jacov (04-09-2012)

  7. #5
    Points: 896, Level: 15
    Level completed: 96%, Points required for next Level: 4

    Posts
    35
    Thanks
    2
    Thanked 3 Times in 3 Posts

    Re: LinReg Assumption on errors ConfIV


    Okay thx

    Let me summarize: In linear regression we have the following results: (always independence assumed)

    : We obtain an "exact" confidence interval for the parameter

    , : We obtain based on the asymptotical normality of the estimator an "approximate" confidence interval for the parameter

    Am I right?

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats