+ Reply to Thread
Results 1 to 12 of 12

Thread: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. regr?

  1. #1
    Points: 46, Level: 1
    Level completed: 92%, Points required for next Level: 4

    Location
    SG, Switzerland
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. regr?




    Hi everyone

    I always thaught that the resulting R^2 of a multiple linear regression with only 1 significant predictor would be equal to the resulting R^2 of a simple linear regression with the same predictor. Is that true, or did I misunderstand something?

    I'd appreciate any help!
    wep

  2. #2
    Omega Contributor
    Points: 38,413, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    7,004
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    So you are saying:


    Simple linear regression with one variable (X1) versus multiple linear regression with X1 plus a group of insignificant covariates in the model?


    The R^2 if not adjusted, will increase with additional variables in the model regardless of their significance. Adding more terms decreases the degrees of freedom in the calculations, so if you have the same term plus more terms, numbers inserted into the formula are not going to be the same in both scenarios.


    P.S., You should run a model for fun to see this.

  3. The Following User Says Thank You to hlsmith For This Useful Post:

    wep (09-10-2015)

  4. #3
    Points: 46, Level: 1
    Level completed: 92%, Points required for next Level: 4

    Location
    SG, Switzerland
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Arrow Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    Thanks for your quick answer! You are of course correct, I'm sorry I did not formulate my question precise enough:

    I am talking about:
    - simple linear regression with one variable (X1)
    versus
    - multiple linear regression with X1 plus a group of other independent variables, but only X1 is kept in the model (backwards entry), because the others are not significant

    Then the R^2 values should be identical, right?
    I am asking because this was always correct in my formerly analyzed data sets and now one data set gives me different R^2 values...

  5. #4
    Devorador de queso
    Points: 95,940, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,937
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    No it typically shouldn't be the same. In general if you add another variable into the model (even an insignificant one) R^2 will either stay the same or increase. It almost always increases (at least slightly). So if you remove these variables the R^2 will decrease.
    I don't have emotions and sometimes that makes me very sad.

  6. #5
    Points: 46, Level: 1
    Level completed: 92%, Points required for next Level: 4

    Location
    SG, Switzerland
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    In case of the multiple linear regression with e.g. X1 + X2 + X3, the R^2 value increased at first, compared to the simple linear regression with X1. But due to the backwards method, X2 and X3 are removed from the model, until only significant predictors remained in the model, in my case only X1. Now the R^2 value should be the same as in the simple linear regression with X1, but that is not true. It is actually lower than in the simple linear regression and I don't understand it.

  7. #6
    Omega Contributor
    Points: 38,413, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    7,004
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    What program are you using?

  8. #7
    Points: 46, Level: 1
    Level completed: 92%, Points required for next Level: 4

    Location
    SG, Switzerland
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    IBM SPSS Statistics 22

  9. #8
    Devorador de queso
    Points: 95,940, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,937
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    Quote Originally Posted by wep View Post
    In case of the multiple linear regression with e.g. X1 + X2 + X3, the R^2 value increased at first, compared to the simple linear regression with X1. But due to the backwards method, X2 and X3 are removed from the model, until only significant predictors remained in the model, in my case only X1. Now the R^2 value should be the same as in the simple linear regression with X1, but that is not true. It is actually lower than in the simple linear regression and I don't understand it.
    Do you have observations that have values missing for x2 or x3 but not X1?
    I don't have emotions and sometimes that makes me very sad.

  10. The Following 2 Users Say Thank You to Dason For This Useful Post:

    hlsmith (09-10-2015), wep (09-10-2015)

  11. #9
    Points: 46, Level: 1
    Level completed: 92%, Points required for next Level: 4

    Location
    SG, Switzerland
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Red face Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    That is the solution!!! Thank you very much, I would never have found out about this myself!

    Because X3 had missing values, these cases were excluded and this stayed like this in the model, even though X3 was removed afterwards.
    If I remove the same cases in X1, I also get the lower value in simple regression, which resolves the whole mistery.

    Thanks again, I was starting to loose it over this...

  12. #10
    Omega Contributor
    Points: 38,413, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    7,004
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    You so smart Dason! Too bad you are a robot.

  13. #11
    Points: 3,015, Level: 33
    Level completed: 77%, Points required for next Level: 35

    Posts
    47
    Thanks
    8
    Thanked 7 Times in 7 Posts

    Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re

    Hi,
    I have fallen into a very similar trap before, and it took me ages to figure out the problem!
    To avoid this trap, I now always look at the "case processing summary" that is generated in SPSS at the start (the bit that I used to just scroll through and ignore).

    If you do this, you will avoid that pitfall!

  14. #12
    Points: 46, Level: 1
    Level completed: 92%, Points required for next Level: 4

    Location
    SG, Switzerland
    Posts
    6
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Wink Re: Is R2 of multiple linear regression with 1 sign. predictor = R2 of simple lin. re


    Yes, and now I will be a bit more careful with the "missing listwise" option in multiple linear regression, especially when applying "backwards method", where variables can be removed from the model.

    Missing data in any of the entered variables also "destroys" the valid data of other variables for the same case.

    Instead "mean substitution" would circumvent this problem, but of course it has other critical aspects.

    Thanks again to all of you, including the robots!

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats