Are they misunderstanding ANOVAs here?

[fishfishfish3]

Hi,

Here's an extract from a paper I'm using in a systematic review:

The mean GAD-7 score of the sample prior to the course was 10.98 (SD 5.17, range 0–21). The mean GAD-7 score after the online mindfulness course was 5.45 (SD 3.77, range 0–19) and at 1 month follow-up 4.60 (SD 3.72, range 0–21). A repeated measures ANOVA indicated that the mean GAD-7 score changed significantly precourse to postcourse (F(1.71,465.4) =325.53, p<0.001) and post hoc tests using the Bonferroni correction revealed that GAD-7 score also further significantly decreased at 1 month follow-up, mean difference post to follow-up 0.850, standard error 0.211, p<0.001.


Does it not indicate that the scores changed significantly at some point between the precourse and follow up measure? Hence the post hoc analysis to find out where this is true?

Thanks for your help,

James

 

[hlsmith]

I read it as the score significantly decreased from pre-course to post course, then they ran a post hoc test to see if it significantly changed form post to 1 month post.

I can't remember how repeated measures ANOVA kicks out its results, so I can't verify the veracity of these statements, however it appears they used the omnibus (f-test) for the first difference then not specified, used a ttest for the latter comparison (ideally dividing the level of significance by 3 to address the 3 possible pairwise post hoc tests). I don't recall, but I would imagine that they should have used a post hoc ttest for that first statistic as well with correction.

 

[noetsi]

Hi,

Here's an extract from a paper I'm using in a systematic review:

The mean GAD-7 score of the sample prior to the course was 10.98 (SD 5.17, range 0–21). The mean GAD-7 score after the online mindfulness course was 5.45 (SD 3.77, range 0–19) and at 1 month follow-up 4.60 (SD 3.72, range 0–21). A repeated measures ANOVA indicated that the mean GAD-7 score changed significantly precourse to postcourse (F(1.71,465.4) =325.53, p<0.001) and post hoc tests using the Bonferroni correction revealed that GAD-7 score also further significantly decreased at 1 month follow-up, mean difference post to follow-up 0.850, standard error 0.211, p<0.001.


Does it not indicate that the scores changed significantly at some point between the precourse and follow up measure? Hence the post hoc analysis to find out where this is true?

Thanks for your help,

James

How is what they are saying [...indicated that the mean GAD-7 score changed significantly precourse to postcourse ....]

different from what you are saying

Does it not indicate that the scores changed significantly at some point between the precourse and follow up measure

Regardless they are saying there was a signficant difference between the pre and post course scores and that they followed this up with a post hoc test to see how they differed [with one rather than all possible pairwise comparisons possibly].

 

[fishfishfish3]

Hey,

Firstly, thank you for your reply. I really appreciate it

It is different because I am saying that over three time points the scores changed according to the ANOVA (pre, post, follow up), whereas they are saying that the scores changed between the first two (pre and post) supported by the significant ANOVA result.

After running the ANOVA they could run three post hoc tests and potentially have found a significant difference between the post and follow up results but not the pre and post. They haven't done this. They have used the ANOVA to justify a difference in pre and post scores and they run a separate post hoc test comparing the post and follow up scores.

Do you see what I mean?

James

 

[hlsmith]

I know exactly what you are saying, do you have a citation to the actual paper?

 

[noetsi]

I have never seen repeated measure ANOVA used for three points (one of them an intermedicate point obviously) at the same time. I was not aware you even could use repeated that way. When I have seen post hoc tests used this has not dealt with repeated measure ANOVA (it was used for 2 or more way ANOVA).

It is interesting to me that you can use post hoc tests this way. I suspect that the authors were not considering this approach.

 

[hlsmith]

I have no idea what they actually did, so that is why I requested the article reference to see if the OP is not reporting some importanat details.

 

[fishfishfish3]

The paper is: Mindfulness online: an evaluation of the feasibility of a web-based mindfulness course for stress, anxiety and depression..

J

 

[hlsmith]

I just skimmed the article and agree that they probably presented their results in a strange and possibly inaccurate way. In particular, the Methods section seems very terse when it comes to analytic approaches.

I would assume that they ran oneway ANOVA with GAD as DV and time period as IV (which from my experience would not be repeated measures ANOVA and would not address covariance from these data not being independent). I attempted to run the analyses based on the parameters presented and got comparable results as the author. An F-test (overall/omnibus test), which was significant. Then post hoc test with bonferroni correction revealed all groups were different, though comparison 2nd vs. 3rd time periods had p=0.0611. I reran this another way and it was signifiicant. Big picture, all three numbers were different, however the last two were close and flirt with significance when using a correction.

However, as menitioned this is not a RM-ANOVA, at least from what they have presented. If all they wanted to do was compare the time periods, they could have run three one-sample ttest using the differences and then corrected their significance level via Bonferroni correct if they wanted.