How to combine experiments using Kruskal-Wallis or Wilcoxon rank sum

Stats newbie/grad student with a question,
I want to compare the statistics from multiple experiments to see if I see the same effect between experiments. Each group within an experiment has an N of 2 or 3, so I can't be sure of normality.
I think Kruskal-Wallis is the appropriate test for comparing multiple groups in this case, is that correct?

Furthermore, some groups are not represented in some experiments. Lastly, the actual numerical values vary wildly between experiments, even when comparing controls to controls, though the intra-experimental variance is reasonable.

I am just learning about the test itself. How do I compare the experiments? Should I combine the Z-values from each experiment, and weight them? Sorry if I am vague, I barely understand this myself.

Thanks, in advance.

With an n of 2 or 3 it is theoretically impossible to acquire a significant p value (<0.05) in a kruskall wallis or mann whitney u (if only 2 experiments) test. The test will run in case of n=3, but the result will never be significant.

Given the very small series of measurements I would just visualize the results by plotting/comparing the median and comparing the range of the measurements.

A side note. Strictly speaking a kruskall wallis test looks for a significant difference between the groups. If you do not find this (p>0.05 or your own choice of alpha) that would indicate that there is no detectable difference. That is statistically NOT analogous to equality. Therefore please consider: are you looking for differences or equality. There are separate tests for equality, but these usually require even more power (i.e. samples) than tests for differences).

Best, MAR
Hello and thank you for the response.
I have about 5 experiments, each with an N of 3. In each experiment there is a positive control group, and 5 treatment groups. Can I express each data set as percentage of positive control, then group the normalized experiments together, so that each treatment group has an N of 15, THEN do the Kruskal test?
I want to see if any one treatment group has a statistically significant higher response (mean) than any others.
Do I understand correctly that your data looks as folllows:
experiment 1
normalized group 1 n=3
normalized group 2 n=3
normalized group 3 n=3
normalized group 4 n=3
normalized group 5 n=3
...and this repeated 5 times (5 experiments)

and that you want to group everything together do that you get one "meta" experiment with:
normalized group 1 n=15
normalized group 2 n=15
normalized group 3 n=15
normalized group 4 n=15
normalized group 5 n=15

My primary issue is with this grouping. To use a kruskall wallis test, the measurements need to be independent. So no technical replicas. I get the impression that the groups of 3 within each group and experiments are more closely related than the groups between the experiments, but that is up to you to decide. If you can motivate that all measurements are equally independent, you can go with a kruskall wallis test.

Best, Martin
Hello and thank you for the response!
1) Yes your interpretation is correct. Each normalized group of N=3 is some cells from a main population. I take an aliquot of cells and add it to group 1, 3 times (N=3). I repeat that for group 2, etc. Each experiment uses identical cells but from a different donor. Thus these are biological replicants rather than technical replicants. All groups within one experiment come from the same original source of immune cells, and between experiments we use a different source of identical cells.

2) Would it be possible to take the average of the average? I heard something about this but know nothing :(
Basically to average the N=3 value for each individual group. Then use these values as individual values for the experiment, so that I get one value per group, per experiment. Then average these already averaged values for an N=5? Could I do statistics on this data do determine if one population mean was statistically greater?

If you are able to recommend a place to start reading online that would bring me from nothing to a good amount of knowledge I would be very very grateful. For now I am reading websites online, but few actually explain how to do things by hand, and what SSQ's and everything are.

Last edited: