+ Reply to Thread
Results 1 to 5 of 5

Thread: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANCOVA

  1. #1
    Points: 12, Level: 1
    Level completed: 23%, Points required for next Level: 38

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANCOVA




    Hi,
    it's the first time I post here. Forgive me if my English is not always correct.
    I want to ask about an incongruity I found when analyzing data from a randomized controlled trial.
    I briefly explain the study: pre-post study to assess the efficacy of an intervention in promoting physical fitness; 1 intervention group and 1 control group; physical fitness assessed pre and post the intervention period, for both groups.

    Now let's come to the problem:
    - running a t-test for intervention group (pre VS post) I found statistically significant difference.
    - running a t-test for control group (pre VS post) I did not find statistically significant difference.
    So, I could say intervention improved physical fitness, while in control group there were no significant changes in physical fitness.
    - but, when running a 2x2 (group x time) repeated measures ANOVA I didn't find any significant difference; the same happened trying to compare gain scores (pre-post change scores) of the 2 groups with a t-test, or when running an ANCOVA on post-scores using pre-scores as covariate.

    Can someone explain this to me? Should I consider the results from rm-ANOVA, gain scores t-test and ANCOVA, since the changes in control group were not significant at first while those in the intervention group where significant?
    Thanks everyone!

  2. #2
    TS Contributor
    Points: 14,811, Level: 78
    Level completed: 91%, Points required for next Level: 39
    Miner's Avatar
    Location
    Greater Milwaukee area
    Posts
    1,171
    Thanks
    34
    Thanked 405 Times in 363 Posts

    Re: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANC

    Did you test the interaction in the 2-way ANOVA?

  3. #3
    TS Contributor
    Points: 17,749, Level: 84
    Level completed: 80%, Points required for next Level: 101
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,540
    Thanks
    56
    Thanked 640 Times in 602 Posts

    Re: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANC

    I briefly explain the study: pre-post study to assess the efficacy of an intervention in promoting physical fitness; 1 intervention group and 1 control group; physical fitness assessed pre and post the intervention period, for both groups.
    How large were your sample sizes? Was allocation to groups
    randomized? What did the "fitness assessment" actually look like?

    - running a t-test for intervention group (pre VS post) I found statistically significant difference.
    - running a t-test for control group (pre VS post) I did not find statistically significant difference.
    So, I could say intervention improved physical fitness, while in control group there were no significant changes in physical fitness.
    What precisely do you mean by significant/nonsignificant?
    0.049 versus 0.050? 0.90 versus 0.00001?

    By the way, "not significant" is not the same as "absent",
    especialy if the statistical power of a study is low and the
    risk for type-2 errors is high.

    - but, when running a 2x2 (group x time) repeated measures ANOVA I didn't find any significant difference;
    What do you mean by this - no significant main effects and no significant
    interaction? What were the exact p-values?

    Generally speaking, the ANOVA tests different questions
    than two dependent samples t-test.
    the same happened trying to compare gain scores (pre-post change scores) of the 2 groups with a t-test, or when running an ANCOVA on post-scores using pre-scores as covariate.
    Well, these ANOVAs try to determine whether the development
    from pre to post was signficantly different between groups.

    Should I consider the results from rm-ANOVA
    Yes. Only this (IMHO) uses the availabe data properly and
    answers the relevant questions.

    With kind regards

    K.

  4. #4
    Points: 12, Level: 1
    Level completed: 23%, Points required for next Level: 38

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANC

    Participants were 10 in intervention group and 11 in control group.
    They were randomly allocated (as I wrote).

    But the point I would like to underline is:
    if there is no statistically significant increase in physical fitness in control group, while in the intervention group there is a significant increase, are further analysis really needed? A logical reasoning suggests no to me... the answer seems to be crystal-clear. But maybe I'm missing something in my reasoning...

    I've got p=0.006 for pre-post in intervention group, while p=0.082 for pre-post in control group (t-test)

  5. #5
    Points: 12, Level: 1
    Level completed: 23%, Points required for next Level: 38

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Incongruity between t-test and 2x2 repeated measures ANOVA/gain scores t-test/ANC


    In the 2x2 repeated measures anova (timexgroup) the within subject effects (time x group) was significant at 0.072

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats