+ Reply to Thread
Results 1 to 13 of 13

Thread: Interaction effect significant but main effects not

  1. #1
    Points: 2,036, Level: 27
    Level completed: 24%, Points required for next Level: 114

    Posts
    28
    Thanks
    34
    Thanked 0 Times in 0 Posts

    Interaction effect significant but main effects not




    Hi all,
    I was wondering if anyone could give me a help with my logit.
    I have a DV of disease vs. non-disease and 3 IV: dummy-year (year 1 vs year 2), dummy-month (5 dummies as I have data for 6 months in each year) and socio-economic status (dummy high vs low).
    They were all significant and so was the interaction year*sociostatus.
    However, when I added the interaction month*year, dummy-year was no longer significant.
    I wanted to estimate the differential effect of having disease between the low socio status people in year 2, so I was going to do =B interaction year*sociostatus - B year.
    How should I proceed with this non-significant coefficient ?
    Thanks a lot!!

  2. #2
    Pirate
    Points: 15,159, Level: 79
    Level completed: 62%, Points required for next Level: 191
    victorxstc's Avatar
    Posts
    875
    Thanks
    229
    Thanked 332 Times in 297 Posts

    Re: Interaction effect significant but main effects not

    It might be a case of multicollinearity, in which a high correlation between at least two independent variables causes strange results. For example a previously-significant variable can become non-significant or sometimes significant but in the reverse direction. Since it is being caused (probably) by a high correlation between a variable and an interaction involving the same variable, you can center that first variable (by subtracting from its mean) to exclude multicollinearity.

    There might be another case, in which adding more variables into the model makes the model as a whole better and more accurate, but makes some of the previously-significant variables non-significant. You should first make sure which one is your case, and then treat it. (I guess your case is likely a case of multicollinearity though).
    "victor is the reviewer from hell" -Jake
    "victor is a machine! a publication machine!" -Vinux

  3. The Following User Says Thank You to victorxstc For This Useful Post:

    PedroR (04-27-2013)

  4. #3
    Points: 881, Level: 15
    Level completed: 81%, Points required for next Level: 19

    Location
    Saint Louis, MO
    Posts
    77
    Thanks
    2
    Thanked 13 Times in 13 Posts

    Re: Interaction effect significant but main effects not

    It's hard for me to imagine what you are describing. In general, just writing out your desired regression equation and then telling us which of the estimates are or are not significant is the most efficient way to enable people to diagnose your problem.

    I would say however, don't worry about an insignificant main effect. If you want to include an interaction term you must include both of the constitutive terms. In other words if you want to interact X1 and X2 then you must also include X1 and X2 separately in your models.

  5. The Following User Says Thank You to threestars For This Useful Post:

    PedroR (05-06-2013)

  6. #4
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Interaction effect significant but main effects not

    Comments I have seen on main effects with signficant interaction question the value of the main effects (or at least the interpretation of them). It's really better to look at simple effects than main effects when interaction is signficant (that is to look at the IV at a specific level of the interacting IV).

    Have you run a VIF/tolerance test to see if multicolinearity is a signficant issue?
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  7. The Following User Says Thank You to noetsi For This Useful Post:

    PedroR (05-06-2013)

  8. #5
    TS Contributor
    Points: 14,811, Level: 78
    Level completed: 91%, Points required for next Level: 39
    Miner's Avatar
    Location
    Greater Milwaukee area
    Posts
    1,171
    Thanks
    34
    Thanked 405 Times in 363 Posts

    Re: Interaction effect significant but main effects not

    I found the following explanation from the University of Oregon, and this corresponds with my past experience. The author describes two classes of interactions, ordinal (lines converge, but do not cross) and disordinal (lines cross). He states that main effects cannot be reliably interpreted for disordinal interactions. He also discusses four approaches for focused tests of interactions. In my experience every instance of significant interaction WITH non significant main effects involved disordinal interactions.

  9. The Following User Says Thank You to Miner For This Useful Post:

    PedroR (05-06-2013)

  10. #6
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Interaction effect significant but main effects not

    I have heard the distinction made between ordinal and disordinal as well although its very common to hear it argued you should not analyze main effects regardless with signficant interaction.

    As always I have no idea who is right
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  11. The Following User Says Thank You to noetsi For This Useful Post:

    PedroR (05-06-2013)

  12. #7
    TS Contributor
    Points: 14,811, Level: 78
    Level completed: 91%, Points required for next Level: 39
    Miner's Avatar
    Location
    Greater Milwaukee area
    Posts
    1,171
    Thanks
    34
    Thanked 405 Times in 363 Posts

    Re: Interaction effect significant but main effects not

    Quote Originally Posted by noetsi View Post
    ...you should not analyze main effects regardless with signficant interaction.
    I would agree with this. In industrial experimentation it is more useful to evaluate the interaction plot than the main effects plots. The main effects plots just indicate general trends.

    The difference between the ordinal and disordinal interactions is primarily due to the factor levels (for continuous factors). This would explain why the significance of a main effect in the presence of a significant interaction may come and go. And why it is important to leave a non-significant main effect in the model when the interaction is significant.

  13. The Following User Says Thank You to Miner For This Useful Post:

    PedroR (05-06-2013)

  14. #8
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Interaction effect significant but main effects not

    Another reason is that the main effects (particularly with disordinal interaction) distort the reality of what is occuring when you have interaction. Thus A might have a stronger effect than B at one level of a interacting IV but B might be stronger than A at another level. To talk about A and B's impact on the dependent variable generally (which is what main effects do) in this situation makes no sense.

    Instead you generate simple effects, the impact of one IV at specific levels of the other IV.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  15. The Following User Says Thank You to noetsi For This Useful Post:

    PedroR (05-06-2013)

  16. #9
    Points: 881, Level: 15
    Level completed: 81%, Points required for next Level: 19

    Location
    Saint Louis, MO
    Posts
    77
    Thanks
    2
    Thanked 13 Times in 13 Posts

    Re: Interaction effect significant but main effects not

    I think this discussion might be confusing what a main effect represents.

    Assume the following model

    Y = b0 + b1*X1 + b2*X2 + b3*X1*X2

    For this model I assume that what we are calling main effects are represented by coefficients b1 and b2. These are also known as constitutive terms. Am I correct? If not, then you should stop reading here!

    But assuming that b1 and b2 are the main effects then it is not correct to say things like, "these represent the general effect." To be clear, b1 represents the effect of X1 on Y when X2 = 0. b2 on the other hand represents the effect of X2 on Y when X1 = 0. This may or may not be substantively interesting, but I would advise creating first difference/marginal effect plots whenever you are using an interaction term.

    The above is easily shown using elementary calculus,

    Partial y/Partial X1 = b1 + b3*X2

    Then if X2 = 0

    Partial y/Partial X1 = b1 + b3*0
    Partial y/Partial X1 = b1

    Thus, the main effect is the effect of X1 on Y when X2 = 0

  17. The Following User Says Thank You to threestars For This Useful Post:

    PedroR (05-06-2013)

  18. #10
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Interaction effect significant but main effects not

    I could be wrong on this but I was taught that (in the absense of interaction) slopes don't reflect the impact of one variable when all the other variables are zero. They reflect the impact of a variable when other variables are held constant regardless of which specific value they are held constant at. It is only when interaction is important that what level they are held constant at matters.

    The intercept B0 is a special case as it is defined as a value when all other X are zero (most commonly).
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  19. The Following User Says Thank You to noetsi For This Useful Post:

    PedroR (05-06-2013)

  20. #11
    Points: 881, Level: 15
    Level completed: 81%, Points required for next Level: 19

    Location
    Saint Louis, MO
    Posts
    77
    Thanks
    2
    Thanked 13 Times in 13 Posts

    Re: Interaction effect significant but main effects not

    Agreed. I was only talking about the case of interaction.

  21. The Following User Says Thank You to threestars For This Useful Post:

    PedroR (05-06-2013)

  22. #12
    Points: 2,036, Level: 27
    Level completed: 24%, Points required for next Level: 114

    Posts
    28
    Thanks
    34
    Thanked 0 Times in 0 Posts

    Re: Interaction effect significant but main effects not

    Thanks for the useful posts. If I understood what you are saying, I must include the constitutive terms in my model also, regardless of their (non-)significance ..
    And in the case of a dummy-variable (created automatically by SPSS), the output has the overall significance of the interaction between one variable (X1) and the coded variable and then the significance for the interactions variable X1-dummy variable 1 and variable -dummy variable 2. If the overall significance is >0,05, should I include the interaction of the variable X1 - dummy variables 1 and 2? And if the overall significance is <0,05, and 1 interaction (variable X1*dummyvariable1) is non-significant, should I include this ? Does that apply to main-effects also ? (when working with dummy variables, include or not to include if the overall significance is non-significant). Sorry for these noob questions and thanks a lot!!

  23. #13
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Interaction effect significant but main effects not


    Thanks for the useful posts. If I understood what you are saying, I must include the constitutive terms in my model also, regardless of their (non-)significance
    If you include the interaction term you do.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats