Simple effects- multiple comparisons adjustments necessary?

[Sunwin]

Hi guys, I cannot seem to figure out what the correct procedure is for simple effects analysis when decomposing interactions.

So I have a significant interaction in a RM ANOVA- I decompose the interaction using simple effects in SPSS (by manipulating the syntax) and I find whats driving the interaction. Does this form of testing require any adjustment? If not, why not?

Found it extremely difficult to find any clear cut answer on this on the web.

 

[hlsmith]

This seems like a nice basic resource:

http://pages.uoregon.edu/stevensj/interaction.pdf

 

[Sunwin]

Thank you for the link. Looks like you need to divide the alpha by the number of comparisons for it to reach significance. However, I have heard of the opposite argument (from my supervisor and in the pdf linked it says that it is 'usually recommended to adjust' which suggests to me that not everyone does and it is not rigorously enforced)- that for subsidiary effects you do not need any correction- is there ANY evidence to back up his claim? Or can I be confident in saying that an adjustment is the best approach? Thank you once again.

 

[hlsmith]

You are using slightly different terminology then I am use to, so not fully following.

You had a significant interaction variable, so you took the continuous independent variable and categorized it (this is common say with age and medical topics) - is this what you did? Now you are wondering if you need to make any adjustments?

 

[Sunwin]

You are using slightly different terminology then I am use to, so not fully following.

You had a significant interaction variable, so you took the continuous independent variable and categorized it (this is common say with age and medical topics) - is this what you did? Now you are wondering if you need to make any adjustments?

Apologies, i'll try to be more clear (although this may confuse you further!). I had a significant interaction variable-yes.

For simplicity's sake, lets say I had a 2 (Cond 1; A, B) X 2 (Cond 2; C, D) repeated measures ANOVA and there was a significant Cond1*Cond2 interaction. By breaking down the ANOVA in SPSS syntax, I then analysed the effects of Cond 1 for C on its own and for D on its own. I found that there was no main effect of Cond 1 for C (p=.8), but there was a significant main effect in D (p=.04). Should the p values be adjusted based on the fact that I ran 2 comparisons?

I hope that this makes sense. If not, ill give it another go at explaining-thank you for taking the time to read this!

 

[hlsmith]

If they were direct pairwise comparisons perhaps. Did you end-up keeping all of the terms in your final model? I see your slight dillema with the p=0.04 increasing.

 

[Sunwin]

I'm not sure what is meant by direct pairwise comparison? I conducted the tests, exactly as described in my previous post and also report them something like this in my final piece of work:

There was a significant Cond1 X Cond2 interaction (F[1,XX]=X, p=.0XX, η2=.XX). This interaction was driven by a significant main effect of Cond 1 (F[1,XX]=X, p=.04, η2=.XX) for D. There was no significant difference in Cond1 for C (F[1,XX]=??, p=.8, η2=.XX)

Does that help? I'm a bit of a stats noob so sorry if i'm not understanding what you're asking.

 

[Sunwin]

anyone make sense of my dilemma? thanks

 

[Sunwin]

makes as much sense as my post lol. spam?