Interpreting my ANOVA results (Interaction effects but lack of main effect)

Kero

New Member
#1
Hi, I'm examining a 2x2 (Time, Gender) mixed design. But for balance reasons, there's also version which would count as an additional IV even though we're not examining it.

My results suggest significant effect with time, with an large effect size (0.6)
Version itself has no significant effect (p=.5) with effect size of (.005)
Yet I'm getting an interaction effect between time and version, just that it's effect size is smaller than the main effect of time (0.2).

Normally I'd be inclined to think that because the effect of time is extremely significant, that an interaction effect will be found when examining time-version purely because of this large significant effect. A type I error perhaps?

At the same time I'm not sure whether this would be wrong to do, and that the data really does suggest that there is an interaction effect, even though one of the IVs by themselves is not significant.

How would one normally interpret this sort of data?
Interaction effects, with only one IV being significant but not the other.
 

Rhodo

New Member
#2
To my knowledge, main effects and interaction effects are independent of each other, meaning interaction effects will not have any influence on main effects and vice versa. Thus a significant main effect, as you describe, would not cause a spurious interaction effect.

If you have an interaction, one plan of action is to look at the simple effects to find out where the interaction is and work on your interpretation from there.
 

Dragan

Super Moderator
#3
To my knowledge, main effects and interaction effects are independent of each other, meaning interaction effects will not have any influence on main effects and vice versa. Thus a significant main effect, as you describe, would not cause a spurious interaction effect.
Note that you're only going to have complete orthoganality if and only if you have equal sample sizes.
 

Kero

New Member
#4
To my knowledge, main effects and interaction effects are independent of each other, meaning interaction effects will not have any influence on main effects and vice versa. Thus a significant main effect, as you describe, would not cause a spurious interaction effect.

If you have an interaction, one plan of action is to look at the simple effects to find out where the interaction is and work on your interpretation from there.
That does make sense. I guess I'm meant to consider that version itself does have an influence when considered with the variable time in that case.

It's just a little more confusing when the results generate significant interaction effects, but not significant main effects.

To re-affirm myself, significant interaction but no main effects would basically suggest that what is obvious, that when the variables are combined there is an effect, but stand alone there is none.

Guess I'll treat it as version does play a factor then (when we were hoping it wouldn't), and that I can't all the data into one group, even if version by itself is non significant.

Note that you're only going to have complete orthoganality if and only if you have equal sample sizes.
Had it not been for the version difference. My original sample size would have had a distribution of about 33 in one group and 37 in another group. (between-subjects).

Since there is an additional version difference.

I'll be forced to have more separate cells where some cells contain about 20 (Which sort of meets the recommended 20 samples per cell requirement) while others contain only 15. Unfortunately since it's not possible to get any more participants, I'm stuck with this data. I can't really tell whether this would be considered as unequal sample sizes in ANOVA, and I'm not sure what it's effects would be on additional research.

Hence my initial query about whether I should treat version has having an effect or not on the data.
 
Last edited:
#5
Hi,

best read this book: 'a student's guide to analysis of variance' by Roberts & Russo. It really sorted out my stats during my phd.

A very common mistake is to think that main effects are more important than interaction effects. It's actually the other way round. If you have an interaction, it means that more interesting things are going on, than just indicated by your main effects.

You could break your study down and run 'simple main effects' analyses on each level of ONE of your independent variables whilst looking at the remaining IV (in plain English, split your data into 2 parts [the 2 levels of your of your IVs] and run 2 seperate anova analyses). Or you could rely posthoc tests to explain the more complex interaction effect.

Read that book though! Superb book! Infact, I think all Psyc faculty members should read it (as lots fall into the 'main effects > interaction effect' trap...).

Cheers,
Andy.

PS not sure I agree with a previous comment saying that main effects and interaction effects are independent. Find a hastily assembled figure attached showing only a few examples of how they interact (see p146 of that book).
 
Last edited: