Problems with using independent t-tests for pre-post intervention study

[LucidMind]

I'm currently reviewing a colleague's paper, and they have an intervention study with 2 groups (therapy vs control) and several outcomes measures taken at pre, post, and a 12 week follow-up. For the data analysis, they looked at changes between pre and post testing (and between pre-test and follow-up testing). They used independent t-tests on the change scores between the two groups, and paired samples t-tests to look at changes within groups.

I think a repeated-measures ANOVA or mixed effects model would be far superior, but i'm having trouble explaining coherent the rationale behind this. I would love to hear thoughts on this...thanks so much in advance!

Edit: There were two groups (Depressed vs Not Depressed) that were randomized to two conditions (obviously the participants couldn't be ethically randomized into the groups). So basically its a 2x2x3 study (depressed vs not depressed x therapy vs control x pre, post, follow-up)

 

[hlsmith]

Was the intervention randomized? If not, they have additional issues if background covariates are imbalanced.

 

[LucidMind]

Was the intervention randomized? If not, they have additional issues if background covariates are imbalanced.

Thanks for the reply. I forgot to mention one detail. There were two groups (Depressed vs Not Depressed) that were randomized to two conditions (obviously the participants couldn't be ethically randomized into the groups).

 

[fffrost]

its a 2x2x3 study (depressed vs not depressed x therapy vs control x pre, post, follow-up)

I completely agree with you. I think the key issues with using multiple t-tests for this are (1) inflated type 1 error rates, and (2) inability to assess interaction effects. I'd be interested to know if there are other reasons too.

For #1 I guess they are making 8 comparisons? So with alpha at 0.05, this means a nearly 40% chance of type 1 error which is pretty bad. Could set alpha to be .05/8, but then ANOVA corrects for that anyway. And as for #2 I'm not sure what interactions we might expect, but I imagine that they would certainly be important to test.