ANOVA or multiple t-tests?

#1
I need to analyse some data (between group differences) that includes four participant groups but I'm not interested in making comparisons between all of them. I.e. I only want to compare group 1 v 2, 1 v 3, 2 v 4 and 3 v 4. So only 4 comparisons rather than the 6 that would be factored into an ANOVA. I know that with more than two groups you should use ANOVA but the power to detect a difference in the comparisons that I'm interested in is impacted upon by having the additional comparisons that I'm not. Would it be preferable to run 4 separate t-tests but then adjust the p-value to <0.0125 to be considered significant? Or can I run ANOVA's and somehow adjust the p-values to account for the fact that I'm ignoring two of the pairwise comparisons? I will also need to assess a change pre- to post-intervention in these groups, which I had planned to use mixed model ANOVA to do although I am again wondering if separate t-tests would be acceptable for the same reasons outlined above. Any advice would be greatly appreciated!
 

obh

Active Member
#2
Hi Jen

Since in the ANOVA the H0 say μ1 = .. = μk, I assume it is better to do the multi comparisons, as this is what you really want to check.
In this case, you should use the Bonferroni correction as you suggested, or probably more accurately the Sidak Correction: Corrected α =0.012741
α = 1-(1-α')^(1/n)