Why use Kruskal-Wallis?


I am putting together a course on inference techniques to use when the predictor
variable(s) are categorical (ie. have no meaningful order). I am wondering if
the Kruskal-Wallis test is still relevant given modern software.

My understanding of K-W is that it is a nonparametric version of 1-way ANOVA that
compares the median of several populations by doing inference on ranks.

My question stems from the facts:
1) ANOVA is known to be quite a robust test. The "rule of thumb" I have seen mentioned is that it is generally okay so long as the largest standard deviation is
not more than twice the smallest standard deviation.
2) Kruskal-Wallis assumes that the various populations have the same shape pdf,
with only the median varying. In particular, it is also sensitive to situations in
which there is a large difference in standard deviations among samples.

Of course, I can imagine situations where there are several populations that
really do have similarly shaped, but non-normal, distributions (which also
cannot be transformed to normal) and differ only in the median, but it seems
that often, when data fails to satisfy the ANOVA conditions, it also doesn't really
satisfy the Kruskal-Wallis assumptions.

It seems like in these situations, probably best practice would be to use some sort
of bootstrapping technique. I don't plan to teach this in my module, but I also want
to be able to mention this if it is true.

I suspect that K-W is perhaps useful in situations where the data is ranked, but
there is not any way to compare differences of ranks, so it is useful because it
does not require making any assumptions about these. However, it still seems
likely that a bootstrapping technique would be better, especially if the
spreads of the various datasets are very different.



any assumptions


Cookie Scientist
I agree, pretty much any situation where Krusal-Wallis might be of use would be better served by a nonparametric bootstrap.

I also note that that Kruskal-Wallis attempts to correct for violations of normality, not for violations of constant variance. Many people seem to think it "fixes everything" but this is simply not true.


No cake for spunky
The 99.95 percent of the population who don't know how to do bootstraping are forced to find alternatives - which is why non-parametrics are used :p
There are strong disagreements whether ANOVA actually is robust (although that is the common wisdom). In particular some argue that heavy tailed distributions can badly distort ANOVA regardless of sample size.
Those are both very helpful. My students, in particular, will fit into that 99.95% of the population who don't know how to
use bootstrapping :) Still, I want them to be aware that better methods exist at a more advanced level.
Also thanks for the note on ANOVA. Do you have a reference for this?


No cake for spunky
I don't have the cite unfortunately. It was written by a Stanford professor (its a book) which I read about 7 years ago. It dealt extensively with limits on ANOVA and recommended strongly bootstraping to address them. I will try to locate it, but its buried admidst tons of other books somewhere. :(


Super Moderator
You're leaving out the important issue of power. That is, the K-W test can be substantially more powerful than the usual parametric ANOVA analysis when the data are skewed and/or heavy-tailed. Studies abound demonstrating this fact.
Excellent, yes that is also helpful. I see the PASS software has power analysis for K-W, but I unfortunately don't
have access to that. I have looked for power calculations for K-W in R using Google, but haven't found anything.
Is there a package that does nonparametric power calculations in R? Also, is there a good place to read about them?
I understand the concept for parametric tests, but I have to confess, I don't really even know how you go about doing
a power test for a nonparametric procedure in principle. I have also just googled "Kruskal-Wallis vs ANOVA power" and
indeed gotten many hits. Any recommendations of a particularly good one to look at?


Super Moderator
Ah, yes, one of those :)
Well, if you manage to excavate it, please let me know!

Have a look at this study. Scroll down to page 205 and see point #4 in their conclusion. They are making the same point that's already known (it's the first time I've seen this study).


Also, you may want to consider the Asymptotic Relative Efficiency (or Pittman Efficiency) of the K-W test to the parametric ANOVA. This establishes the power advantage of the K-W analytically. For example, when sampling from a Double Exponential distribution the K-W test is 1.5 times more powerful than the F-test. When sampling from an Exponential distribution the K-W test is 3.0 times more powerful than the ANOVA F -test. For more on this see Conover (1999, p. 297). Practical Nonparametric Statistics (3rd).


No cake for spunky
I looked at gpower (the one most recommended for power analysis at my program) and while it will calculate power for Wilcoxxon I don't see one for KW.