Mixed Model ANOVA or MLM for small sample?

#1
Hi there!

So I just recently finalized my dissertation analyses and sent them to my advisor, and (as this process goes) he is now "recommending" that I change my analytic plan. I could use some advice.

My dissertation is a small pilot/feasibility study for a newly developed intervention. The design is simple, looking at responses to 4 separate self-report measures at pre- and post-intervention. Despite extensive efforts, I was only able to recruit 23 participants, who were randomized into either the intervention (n=12) or waitlist control (n=11). Over the course of the intervention, 3 dropped out of the intervention and 2 from the waitlist, leaving me with 18 post-intervention (9 in each group). My analytic plan is simple - a series of mixed model 2(time) X 2(group) ANOVAs for each of the four outcome variables (this was bumped down from a MANOVA given my small sample and only moderate correlations between the outcome variables). My advisor is now wanting me to conduct intention to treat analyses. Long story short, I consulted with a statistician who said that imputing missing post-treatment values for drop outs with such a small sample is not recommended, and he suggested doing multilevel modeling if I need to run ITT analyses.

My question - Which analytic plan is better, given my small sample and relatively simple study design (just two time points - pre and post)? I understand that MLM will allow me to include all of the baseline data regardless of drop outs, but my gut tells me that MLM would require more power given that it is more complex (and I barely have enough for ANOVAs given my tiny sample size).

And my follow-up - are there any good references I can cite that support my (your) choice? Say, if I ever need to DEFEND my decision in the near future...:)


Thanks so much!
 

Jake

Cookie Scientist
#2
I understand that MLM will allow me to include all of the baseline data regardless of drop outs, but my gut tells me that MLM would require more power given that it is more complex
This is an understandable intuition but it is actually not the case. In all likelihood a multilevel model will actually have greater power to detect a pre-post difference in your dataset, not less power.

By the way, strictly speaking it does not make sense to say that one method "requires more power" than another method. What does make sense, and what I assume you meant to say, is that some methods might require a greater sample size just to achieve the same power as another method. But again, this is not true in the present case.