+ Reply to Thread
Results 1 to 9 of 9

Thread: Testing for difference 2 regression models

  1. #1
    Points: 2,804, Level: 32
    Level completed: 36%, Points required for next Level: 96

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Testing for difference 2 regression models




    Hello,

    I am working on multiple regressions at the moment and have come to a point that I need a little assistance. I have the following question:

    Is there a way to test for significant difference between 2 regression models?
    That would mean testing for the difference between 2 R-Squared values?
    If so, which test should I use? (partial F-test or F-test?). Assuming this cannot be done in SPSS, how does it work manually?

    I know these are actually 3 questions but could someone help me out with this predicament (another question)?

    Thanks so much!

  2. #2
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts
    Quote Originally Posted by devreede View Post
    Hello,

    I am working on multiple regressions at the moment and have come to a point that I need a little assistance. I have the following question:

    Is there a way to test for significant difference between 2 regression models?
    That would mean testing for the difference between 2 R-Squared values?
    If so, which test should I use? (partial F-test or F-test?). Assuming this cannot be done in SPSS, how does it work manually?

    I know these are actually 3 questions but could someone help me out with this predicament (another question)?

    Thanks so much!

    I think the Chow test might be helpful to you.

    http://www.stata.com/support/faqs/stat/chow.html

  3. #3
    Points: 3,504, Level: 37
    Level completed: 3%, Points required for next Level: 146

    Posts
    139
    Thanks
    1
    Thanked 1 Time in 1 Post
    That was great fun - I spent the day experimenting with that

    Also try
    http://pages.infinit.net/rlevesqu/Sy...e/ChowTest.txt

    It's kinda useful to have two approaches to make sure you're doing it right ...
    When they finally line up it's fantastic!

  4. #4
    Points: 2,804, Level: 32
    Level completed: 36%, Points required for next Level: 96

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    I think I mean something else...

    Thanks a lot for your answers although my problem is slightly different, let me explain in more detail:

    I ran a regression equation with 25 variables. Subsequently, I removed 5 of those variables and ran the equation again. I would like to test whether the R-squared of the first regression equation is significantly different form the second regression equation.

    How do I go about doing this and which test doe I use?

    Thanks again people!

  5. #5
    Points: 3,504, Level: 37
    Level completed: 3%, Points required for next Level: 146

    Posts
    139
    Thanks
    1
    Thanked 1 Time in 1 Post
    No ... the Chow test is indeed the way to go ...

    I think the link I sent you is the easiest one to follow for the kinda chow you want ...

    You'll have two grouping variables ... let's call then 20 and 25 to reflect the number of variables - so, a big long column of 20s and that followed by a bunch more 25s. That's your fixed factor in the univariate anova.

    Then your DV can be whatever the outcome predictor was - following the regression. A single column of outcomeiness. let's call it outcome.

    Then you co-variate is whatever you were trying to predict.

    Use paste instead of OK

    To the syntax design add *outcome (the last line)

    Run all
    (actually - read the link carefully on this ...)

    Then you'll get a F and a P value.

    Booya ... any sig diff between the two models.

  6. #6
    Points: 3,504, Level: 37
    Level completed: 3%, Points required for next Level: 146

    Posts
    139
    Thanks
    1
    Thanked 1 Time in 1 Post
    Oh - here's what you might also be describing ...
    Assume SPSS 12 (it's all I speak)

    Put the first 20 variables into the regression using the ENTER METHOD and click next
    Put the remaining 5 variables in - again ENTER
    Make sure you opt for r-square change
    This is a hierarchical multiple regression

    Check the model summary for results.
    The sig F change will tell you whether those 5 extra were ... significantly helpful

    (a little bit of a different question though)

  7. #7
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts
    Quote Originally Posted by Philyuko View Post
    Oh - here's what you might also be describing ...
    Assume SPSS 12 (it's all I speak)

    Put the first 20 variables into the regression using the ENTER METHOD and click next
    Put the remaining 5 variables in - again ENTER
    Make sure you opt for r-square change
    This is a hierarchical multiple regression

    Check the model summary for results.
    The sig F change will tell you whether those 5 extra were ... significantly helpful

    (a little bit of a different question though)
    By hand calculation it is:

    F = [(R^2_Full - R^2_Reduced) / (25 - 20) ] / [ (1-R^2_Full) / ( N - 25 - 1) ]

    where R^2_Full is associated with the model with 25 I.V.'s and
    R^2_Reduced is associated with the model with 20 I.V.'s.

    N is the total sample size (which I hope is large).

  8. #8
    Points: 3,504, Level: 37
    Level completed: 3%, Points required for next Level: 146

    Posts
    139
    Thanks
    1
    Thanked 1 Time in 1 Post
    Dragan

    Do you know anything about a Chow test for catagoricals?

    I typically deal with discriminate analysis rather than MR ...

    any thoughts?

    I guess the bottom line is sig diffs between F1 values because that's what I end up with.

  9. #9
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Quote Originally Posted by Philyuko View Post
    Dragan

    Do you know anything about a Chow test for catagoricals?

    Not off "the top of my head"....I'll look around.

+ Reply to Thread

           




Similar Threads

  1. Adj-r2 from two models and the significance of the difference
    By mister_bluesman in forum Statistics
    Replies: 4
    Last Post: 07-08-2013, 09:00 PM
  2. Replies: 5
    Last Post: 08-28-2010, 08:46 PM
  3. testing no difference
    By vilas in forum Statistics
    Replies: 0
    Last Post: 12-14-2009, 01:04 PM
  4. Specification Testing for Spatial Models
    By gatormka in forum Statistics
    Replies: 0
    Last Post: 12-15-2008, 01:26 PM
  5. Testing Regression Models - sample size
    By ajlobo in forum Psychology Statistics
    Replies: 0
    Last Post: 03-17-2008, 02:02 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats