+ Reply to Thread
Results 1 to 9 of 9

Thread: Are they misunderstanding ANOVAs here?

  1. #1
    Points: 15, Level: 1
    Level completed: 29%, Points required for next Level: 35

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Question Are they misunderstanding ANOVAs here?




    Hi,

    Here's an extract from a paper I'm using in a systematic review:

    The mean GAD-7 score of the sample prior to the course was 10.98 (SD 5.17, range 021). The mean GAD-7 score after the online mindfulness course was 5.45 (SD 3.77, range 019) and at 1 month follow-up 4.60 (SD 3.72, range 021). A repeated measures ANOVA indicated that the mean GAD-7 score changed significantly precourse to postcourse (F(1.71,465.4) =325.53, p<0.001) and post hoc tests using the Bonferroni correction revealed that GAD-7 score also further significantly decreased at 1 month follow-up, mean difference post to follow-up 0.850, standard error 0.211, p<0.001.


    Does it not indicate that the scores changed significantly at some point between the precourse and follow up measure? Hence the post hoc analysis to find out where this is true?

    Thanks for your help,

    James

  2. #2
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Are they misunderstanding ANOVAs here?

    I read it as the score significantly decreased from pre-course to post course, then they ran a post hoc test to see if it significantly changed form post to 1 month post.

    I can't remember how repeated measures ANOVA kicks out its results, so I can't verify the veracity of these statements, however it appears they used the omnibus (f-test) for the first difference then not specified, used a ttest for the latter comparison (ideally dividing the level of significance by 3 to address the 3 possible pairwise post hoc tests). I don't recall, but I would imagine that they should have used a post hoc ttest for that first statistic as well with correction.
    Stop cowardice, ban guns!

  3. #3
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Are they misunderstanding ANOVAs here?

    Quote Originally Posted by fishfishfish3 View Post
    Hi,

    Here's an extract from a paper I'm using in a systematic review:

    The mean GAD-7 score of the sample prior to the course was 10.98 (SD 5.17, range 0–21). The mean GAD-7 score after the online mindfulness course was 5.45 (SD 3.77, range 0–19) and at 1 month follow-up 4.60 (SD 3.72, range 0–21). A repeated measures ANOVA indicated that the mean GAD-7 score changed significantly precourse to postcourse (F(1.71,465.4) =325.53, p<0.001) and post hoc tests using the Bonferroni correction revealed that GAD-7 score also further significantly decreased at 1 month follow-up, mean difference post to follow-up 0.850, standard error 0.211, p<0.001.


    Does it not indicate that the scores changed significantly at some point between the precourse and follow up measure? Hence the post hoc analysis to find out where this is true?

    Thanks for your help,

    James
    How is what they are saying [...indicated that the mean GAD-7 score changed significantly precourse to postcourse ....]

    different from what you are saying

    Does it not indicate that the scores changed significantly at some point between the precourse and follow up measure
    Regardless they are saying there was a signficant difference between the pre and post course scores and that they followed this up with a post hoc test to see how they differed [with one rather than all possible pairwise comparisons possibly].
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  4. #4
    Points: 15, Level: 1
    Level completed: 29%, Points required for next Level: 35

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Are they misunderstanding ANOVAs here?

    Hey,

    Firstly, thank you for your reply. I really appreciate it

    It is different because I am saying that over three time points the scores changed according to the ANOVA (pre, post, follow up), whereas they are saying that the scores changed between the first two (pre and post) supported by the significant ANOVA result.

    After running the ANOVA they could run three post hoc tests and potentially have found a significant difference between the post and follow up results but not the pre and post. They haven't done this. They have used the ANOVA to justify a difference in pre and post scores and they run a separate post hoc test comparing the post and follow up scores.

    Do you see what I mean?

    James

  5. #5
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Are they misunderstanding ANOVAs here?

    I know exactly what you are saying, do you have a citation to the actual paper?
    Stop cowardice, ban guns!

  6. #6
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Are they misunderstanding ANOVAs here?

    I have never seen repeated measure ANOVA used for three points (one of them an intermedicate point obviously) at the same time. I was not aware you even could use repeated that way. When I have seen post hoc tests used this has not dealt with repeated measure ANOVA (it was used for 2 or more way ANOVA).

    It is interesting to me that you can use post hoc tests this way. I suspect that the authors were not considering this approach.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  7. #7
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Are they misunderstanding ANOVAs here?

    I have no idea what they actually did, so that is why I requested the article reference to see if the OP is not reporting some importanat details.
    Stop cowardice, ban guns!

  8. #8
    Points: 15, Level: 1
    Level completed: 29%, Points required for next Level: 35

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Are they misunderstanding ANOVAs here?

    The paper is: Mindfulness online: an evaluation of the feasibility of a web-based mindfulness course for stress, anxiety and depression..

    J

  9. #9
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Are they misunderstanding ANOVAs here?


    I just skimmed the article and agree that they probably presented their results in a strange and possibly inaccurate way. In particular, the Methods section seems very terse when it comes to analytic approaches.

    I would assume that they ran oneway ANOVA with GAD as DV and time period as IV (which from my experience would not be repeated measures ANOVA and would not address covariance from these data not being independent). I attempted to run the analyses based on the parameters presented and got comparable results as the author. An F-test (overall/omnibus test), which was significant. Then post hoc test with bonferroni correction revealed all groups were different, though comparison 2nd vs. 3rd time periods had p=0.0611. I reran this another way and it was signifiicant. Big picture, all three numbers were different, however the last two were close and flirt with significance when using a correction.

    However, as menitioned this is not a RM-ANOVA, at least from what they have presented. If all they wanted to do was compare the time periods, they could have run three one-sample ttest using the differences and then corrected their significance level via Bonferroni correct if they wanted.
    Stop cowardice, ban guns!

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats