+ Reply to Thread
Results 1 to 3 of 3

Thread: Diff p-value when using 1wayANOVA and 2wayANOVA on same set of data

  1. #1
    Points: 5, Level: 1
    Level completed: 9%, Points required for next Level: 45

    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Diff p-value when using 1wayANOVA and 2wayANOVA on same set of data




    I used the 2way ANOVA to compare 2 treatments (control vs. T1) across 4 groups (A-D). I used the 1way ANOVA to compare 3 treatments (control vs. T1 vs. T2) in group A - only group A had 3 treatments.


    But the results for the tests were different: the 2way ANOVA said there was no effect of T1 in group A, but the 1way comparing the 3 treatments said there was an effect between control and T1. Both used Bonferroni multiple comparison's test, and compared the cell mean with each row, same confidence interval. 2way ANOVA p-value was 0.4496, 1way ANOVA p-value was 0.0348.


    What's going on, shouldn't the p-value be the same since the data is the same and the test is essentially testing the same thing? Am I missing something that is fundamental?

  2. #2
    Human
    Points: 12,676, Level: 73
    Level completed: 57%, Points required for next Level: 174
    Awards:
    Master Tagger
    GretaGarbo's Avatar
    Posts
    1,362
    Thanks
    455
    Thanked 462 Times in 402 Posts

    Re: Diff p-value when using 1wayANOVA and 2wayANOVA on same set of data

    Maybe the data was not balanced.

    What's going on, shouldn't the p-value be the same since the data is the same and the test is essentially testing the same thing? Am I missing something that is fundamental?

    No, the data is not the same and the model is not the same.

  3. #3
    Points: 3,506, Level: 37
    Level completed: 4%, Points required for next Level: 144

    Posts
    129
    Thanks
    3
    Thanked 26 Times in 26 Posts

    Re: Diff p-value when using 1wayANOVA and 2wayANOVA on same set of data


    First of all, was the group by treatment interaction from the 2 way significant? If it was NOT, then you shouldn't even be looking at simple effects (like t1 versus control within each group). It is the presence of the interaction that supports and justifies such comparisons.

    Assuming there was one, did the simple effects from the 2-way ANOVA use the pooled error term from all 8 group by treatment combinations (the MS within) as a basis for the error term? If so, might some of the other groups have had much greater variability than group A? That could increase the error term and make it harder to see differences. Similar issue if this was a repeated measures design.

    With a 2-group comparison, the program may not apply any Bonferroni correction. With 3 groups, it would correct. So that could affect the p-value as well.

    You saw the CI was the same. Do you mean the CI for the difference of means? If those are the same in the two analyses, then the p-values should be the same. But I doubt that is what you meant.

    Ed

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats