Interaction Term Statistically Significant in Full Model, but not Reduced

[ryc]

As a preface, I had been taught to assess the interaction term prior to higher end analyses. Figuring out whether or not you have to stratify your analysis early on is essential to saving time in the long run.

I am in the process of finalizing a manuscript (which has taken about a year to synthesize) and have been asked by my one of my co-authors if I evaluated an interaction term for a series of ordinal logistic regression models. Proportional odds assumption is valid, and my exposure is continuous. My dilemma exists in that I HAD evaluated the interaction term during my initial model building phases and it came up non-significant (p = 0.384).

eqn: y = B(group)+ B(exposure) + B(group*exposure) + baselinelogit)

However, since it had been so long since I did this, to satisfy their curiosity, I added the interaction to the full model after model building was long completed and it was statistically significant (p = 0.056).

eqn: y = B(group) + B(exposure) + B(group*exposure) + B(age) + B(followup_time) + baselinelogit)

I am a bit befuddled as to whether or not I should report the interaction term as significant now.

I only have about 360 observations in my dataset, but with only 5 terms, I doubt I'm overspecifiying the model. In addition, after stratifying by group, there is an independent association between the exposure and the outcome in one group but not the other.

I hope I was descriptive enough. Thanks in advance for any help you can offer.

 

[Tom S]

To be honest, I don't know the answer to your question, and I'm afraid I would get a headache if I tried to figure it out. But I would like to make one comment on your post.

Your first paragraph suggests that your choice of analysis depends on that preliminary test for interaction. To me, that implies that if the interaction is "large" or "great" or whatever term you would like to use, then you would use a different analysis from the one you would employ if the interaction is "small." Does a significance test tell you that? I don't think so. A significant interaction might be so small as to be uninteresting while a large interaction might not prove statistically significant. The problem is, if you want to base a decision on the interaction, I think you need to think about the power of that interaction test.

Well, good luck with that. Interactions are hard to think about. While you could probably envision what no interaction is, we need to think about other "values" of the interaction and consider the power of the test for some alternative values. Now that really gives me a headache.

This is one of those many examples, I think, where statisticians practice things differently from how we tell our students they should.