Effect size for small non-parametric data

#1
Hi Guys,

I would like to run a effect size test for a very small non-parametric analysis (n=4 per group), which one is the most recommended test?

Thank you in advance.
 

Karabiner

TS Contributor
#2
I would like to run a effect size test for a very small non-parametric analysis (n=4 per group), which one is the most recommended test?
If you mean significance test, then the recommendation depends
on the number of groups, the scale level of the data, the research
question, and design features such as: are these independent or
dependent measures.

With kind regards

K.
 
#3
Hi Karabiner, Thanks for your kind reply.

Let me try to be more specific. I have two groups (n=4, before-after same subject), using Wilcoxon matched-pairs signed rank test my p-value was not significant.
Seems very likely that my analysis is underpowered by small sample size. I would like to use a test to estimate my effect size. Just to exemplify, when my data is parametric, I usually perform paired t-test associate with Cohen's D. I would like to know which test would be the non-parametric surrogate for Cohen's D.

Thanks for your time and attention.

Kind regards.
 

CowboyBear

Super Moderator
#4
Could you describe your dependent variable?

Effect sizes do not necessarily have to be standardized. You could simply report the mean difference on whatever your DV is. If the DV has any kind of meaningful unit of measurement, this is probably the most informative thing to do. If you really want a standardized measure, you could just stick with Cohen's D (it doesn't have a normality assumption, or use something designed for ordinal data like somers d or kendall's tau.
 
#5
Hi CowboyBear,

Many thanks for your reply. My dependent variable is the number of bacteria in a specific material, before and after the exposure to the human saliva.

Kind regards
 

CowboyBear

Super Moderator
#6
Hi CowboyBear,

Many thanks for your reply. My dependent variable is the number of bacteria in a specific material, before and after the exposure to the human saliva.

Kind regards
I would say that the most direct and meaningful effect size would simply be a mean difference, or maybe something the percentage reduction in number of bacteria occurring after the exposure to human saliva. Standardized effect sizes like Cohen's d are popular especially in fields like psychology where the variables used often don't have meaningful units of measurement, so we use the standard deviation as a unit of measurement instead. Your DV does have very meaningful units, so there seems to be no obvious reason to add the complexity of a standardized effect size. That said, if you really wanted to report Cohen's D, there is no obvious statistical reason why it would be inappropriate. Cohen's D does not require the assumption of normal distributions within each level of the IV.