Significant Friedman's test but non-significant post-hoc tests

C

Christy127

New Member
#1
#1
Hello,

I am running a Friedman's test because I want to see if there is any difference in my participants' scores across 4 time-points. My sample is n=51. I ran the test and it revealed a statistically significant difference (p = 0.29). Then I conducted post hoc tests to see where the difference lies. I also used a Bonferroni adjustment which is 0.05/6 = 0.008. However, the results from the post-hoc tests were not significant, that is, were higher than 0.008. I have attached a screenshot from SPSS so that you can take a look. I wonder if that's because my sample is too small. How do I report such a finding?

Thank you very much in advance.
 

Attachments

L

liweitang

New Member
#2
#2
the logic of the ANOVA (with some assumptions) is doing the F test first, If we reject the ANOVA hypothesis, then we should do the pairwise comparisons, such as Bonferroni correction or Tukey’s Studentized Range test.
 
O

obh

Active Member
#5
#5
Generally, the Bonferroni correction may sometimes be too strict reducing type 1 error. (reducing the probability to reject a correct H0) with the price of increasing type 2 error (increasing the probability to accept an incorrect H0. I would use the Tukey HSD instead.

But it is also correct to use the Bonferroni correction!
 
Karabiner

Karabiner

TS Contributor
#7
#7
As far as I can see, also your uncorrected p-values are not statistically significant on the 5% level.

Personally, I do not make correction in post hoc tests if the global test is statistically significant,
but opinions certainly differ.

With kind regards

Karabiner
 
C

Christy127

New Member
#8
#8
Karabiner said:
Personally, I do not make correction in post hoc tests if the global test is statistically significant
Click to expand...
Do you mean that you don't apply the Bonferroni correction in your post hoc tests? So you use the ones that are below the 5% level? I only apply the corrections because everybody else in my field does it, so I just follow the norm...

Another thing that I tried is I conducted Wilocoxon tests separately and there I found statistically significant results. It's as if the option on the SPSS (that's on the screenshot I uploaded) does not give significant results, but when I conduct the Wilcoxon tests independently, there are statistically significant results also even after I've applied the Bonferroni correction. It's as if using two different options on SPSS gives you different results...
 
Karabiner

Karabiner

TS Contributor
#9
#9
The Wilcoxon signed rank test uses data more efficiently than Friedman,
but it requires an interval scaled dependent variable. Since you used
Friedman, I supposed that your data is only ordinal.

With kind regards

Karabiner
 
C

Christy127

New Member
#10
#10
No, my data is continuous, but non-normally distributed. That's why I used Friedman. I thought that Wilcoxon tests are the post-hoc tests after I conduct the Friedman's test, to check were the difference lies if the test gives me a significant result.

Thank you.
 
Karabiner

Karabiner

TS Contributor
#13
#13
With n=51 you could consider repeated-measures ANOVA. It is robust against nonnormal residuals
with a sample size as large as this (see central limit theorem).

With kind regards

Karabiner
 
You must log in or register to reply here.