# Thread: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANCOVA

1. ## Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANCOVA

Hi,
it's the first time I post here. Forgive me if my English is not always correct.
I want to ask about an incongruity I found when analyzing data from a randomized controlled trial.
I briefly explain the study: pre-post study to assess the efficacy of an intervention in promoting physical fitness; 1 intervention group and 1 control group; physical fitness assessed pre and post the intervention period, for both groups.

Now let's come to the problem:
- running a t-test for intervention group (pre VS post) I found statistically significant difference.
- running a t-test for control group (pre VS post) I did not find statistically significant difference.
So, I could say intervention improved physical fitness, while in control group there were no significant changes in physical fitness.
- but, when running a 2x2 (group x time) repeated measures ANOVA I didn't find any significant difference; the same happened trying to compare gain scores (pre-post change scores) of the 2 groups with a t-test, or when running an ANCOVA on post-scores using pre-scores as covariate.

Can someone explain this to me? Should I consider the results from rm-ANOVA, gain scores t-test and ANCOVA, since the changes in control group were not significant at first while those in the intervention group where significant?
Thanks everyone!

2. ## Re: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANC

Did you test the interaction in the 2-way ANOVA?

3. ## Re: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANC

I briefly explain the study: pre-post study to assess the efficacy of an intervention in promoting physical fitness; 1 intervention group and 1 control group; physical fitness assessed pre and post the intervention period, for both groups.
How large were your sample sizes? Was allocation to groups
randomized? What did the "fitness assessment" actually look like?

- running a t-test for intervention group (pre VS post) I found statistically significant difference.
- running a t-test for control group (pre VS post) I did not find statistically significant difference.
So, I could say intervention improved physical fitness, while in control group there were no significant changes in physical fitness.
What precisely do you mean by significant/nonsignificant?
0.049 versus 0.050? 0.90 versus 0.00001?

By the way, "not significant" is not the same as "absent",
especialy if the statistical power of a study is low and the
risk for type-2 errors is high.

- but, when running a 2x2 (group x time) repeated measures ANOVA I didn't find any significant difference;
What do you mean by this - no significant main effects and no significant
interaction? What were the exact p-values?

Generally speaking, the ANOVA tests different questions
than two dependent samples t-test.
the same happened trying to compare gain scores (pre-post change scores) of the 2 groups with a t-test, or when running an ANCOVA on post-scores using pre-scores as covariate.
Well, these ANOVAs try to determine whether the development
from pre to post was signficantly different between groups.

Should I consider the results from rm-ANOVA
Yes. Only this (IMHO) uses the availabe data properly and

With kind regards

K.

4. ## Re: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANC

Participants were 10 in intervention group and 11 in control group.
They were randomly allocated (as I wrote).

But the point I would like to underline is:
if there is no statistically significant increase in physical fitness in control group, while in the intervention group there is a significant increase, are further analysis really needed? A logical reasoning suggests no to me... the answer seems to be crystal-clear. But maybe I'm missing something in my reasoning...

I've got p=0.006 for pre-post in intervention group, while p=0.082 for pre-post in control group (t-test)

5. ## Re: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANC

In the 2x2 repeated measures anova (timexgroup) the within subject effects (time x group) was significant at 0.072

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts