Do you have different sample sizes for group A and B? Are the standard deviations similar for all groups?
The general question I have is, what do I do when a post-hoc difference comparison of 2 means form an ANOVA (that has a total of 4 groups) shows no statistical difference, but looking at the raw data, the means of those groups, the stdev of those groups, and running a t-test on them indicates they are different?
The details are as follows: I recently ran an ANOVA looking at the effect of treatment or placebo on group A and group B mean values of a hormone. The ANOVA post-hocs indicated significant differences in treated vs placebo for group B, but not group A. But Group A shows the bigger difference in means from treatment versus placebo. Also, Group A (treated or placebo) show tighter stdev than the Group B values. Finally, separate t-tests for Groups A and B indicate significant treatment effects.
I'm not sure which statistics to trust here. I originally went with the ANOVA (using Statistica), but the raw data and mean values just seem too different for Group A and the t-test result seems more believable based on this.
Thoughts?
Do you have different sample sizes for group A and B? Are the standard deviations similar for all groups?
A: Groups are n=15 (placebo) and n=11 (treat). Stdev are 9 and 10
B: Groups are n=15 (placebo) and n=11 (treat). Stdev are 33 and 57
Group A means are 57 and 29
Group B means are 289 and 233
Group A are genetically different than Group B so we expect them to be lower regardless of treatment and they clearly are. ANOVA confirmed this as well.
Do you know the assumptions that Anova makes?
http://en.wikipedia.org/wiki/Analysi...tions_of_ANOVA
Anova is assuming all groups have an equal variance.
Right, but this assumption is less important when sample sizes are similar. Do you you think 11 and 15 are not similar enough?
How exactly are you running your ANOVA? Are you running an analysis on group A and an independent one on group B?
Yes, but the assumption of equal variances is less important when you have 1) similar sample sizes or 2) large sample sizes. I don't have the latter, but I do have the former...though it could be argued that 11 and 15 are not similar enough.
I am running one anova. Group A placebo, Group A treat, Group B placebo, Group B treat. 4 groups total. Group A placebo and treat are the ones that aren't coming out different and yet they have the tighter data.
If you're doing one analysis, then you're violating that assumption. I don't think it matters that you have similar sample sizes. The std dev's in groups A and B are too different. My impression is that the t-test is the way to go.
Thanks guys. I just log transformed the data and ran the ANOVA and it showed the differences I was expecting (and confirmed the t-test results too).
I was going to recommend a log transform. Were you doing a Welch t-test before?
Tweet |