General two-way ANOVA questions

[sfishe]

Hi,

I have the following factors in my data: pretreatment (yes or no), drug_dose (1, 2, 3, 4) and neurochemical levels (dependent variable).

I'm doing a two-way ANOVA using GraphPad, and I have SPSS at my disposal as well.

For the data I'm interested in, I'm getting a significant main effect of pretreatment but a nonsignificant interaction effect and a nonsignificant drug_dose effect. Using the Bonferroni posttest in GraphPad I also get significant results in comparisons between two of the drug doses in terms of pretreatment value.

I have a few questions related to this scenario. Firstly, is it statistically valid to do any posthoc/multiple comparison tests after a two-way ANOVA if there is not a statistically significant interaction effect?

Second, if I can do posthoc tests with just a single significant main effect, do these particular Bonferroni results described above mean anything? I.e. are they valid (e.g. without an interaction)?

Thanks for your help!

[ichbin]

No, it is not acceptable to do post-hoc tests on individual factors if the ANOVA says the factor is not significant. The reason is that doing so invites precisely the problem that ANOVA was designed to solve: that given many individual tests, the aggregate chance for a false positive increases greatly.

Actually, though, I don't think an ANOVA is the best analysis vehicle for the setup you describe. The reason is that drug_dose is actually a continuous, not a categorical variable. By treating it as a categorical variable, you are making it harder for your analysis to detect its effect: your test would have to see a clear step between two levels, rather than a trend of small changes between each level, because it does not know that the categories represent increasing levels.

I think a multilinear regression would be a better analysis vehicle. It will have no problem with your pretreatment variable, because a binary category can be safely encoded as a 0 or 1 value.

[sfishe]

ichbin,

Thanks for the reply. Just to clarify, a significant interaction effect is required to do any post-hoc tests?

I have significant main effects for factors, but no interaction...

And thanks, I'll check out multilinear regression.

[ichbin]

1. I got a different impression of your factor significance situation from your first post. There you said "I'm getting a significant main effect of pretreatment but a nonsignificant interaction effect and a nonsignificant drug_dose effect". But in your reply post you said "I have significant main effects for factors, but no interaction", which makes is sound like you have a significant drug_dose effect. Which is it?

2. If you do have a significant drug_dose effect, they you certainly may do t-tests on it. Having no significant interaction effect is not a problem; in fact, it makes your conclusions clearer and cleaner.

3. I do still recommend looking at multilinear regression as a superior way to analyze this data.

[sfishe]

Sorry for the confusion - I have a lot of analyses I'm working through at the moment. In the example I was thinking of, I do only have one significant main effect: pretreatment.

So you seemed to be saying that I can only do further posttests (Tukey or Bonferroni etc.) if I have a significant drug_dose effect? (or interaction obviously), which would be to find which doses differ significantly from each other, right?

In regards to a sig. pretreatment effect: I know the idea that doing posttests on a factor with 2 levels doesn't 'make sense', but I still don't quite see how I can further investigate at which doses there is a significant difference between pretreatment and no pretreatment...which isn't this what posthoc tests are for in general? E.g. at dose 1, does pretreatment (yes or no) affect the neurochemical levels. Having a general sig. effect for pretreatment out of the anova doesn't tell me this does it.

Thanks a lot for your help - sorry I'm at the awkward 'lots of bits of info but no clear picture' stage of understanding all this