comparing means\unequal(small) size group

#1
Hi,
I am about to use student t test to compare the mean of two independent groups. The first group sample size is 4 and another one is 24. Can I use t-test? Is there any other test? shall I use a nonparametric test?
Regards,
 

gianmarco

TS Contributor
#2
t-test

Hi!

I think that the main concern (in order to use t-est or other parametric tests) is whether or not your samples are normal distributed AND have the same variance.

If they are normal distributed AND have the same variance, use the t-test;
if they are normal distributed BUT without same variance, use Welch's t;
if they are not normal distributed (and I suspect that is holds true at least for the small sample), use a non-parametric test (Mann-Whitney, Kolmogorov-Smirnov).

You can read this link (http://onlinestatbook.com/index.html) about t-test, or other link available in the web. Is very useful this: http://faculty.vassar.edu/lowry/VassarStats.html.
It provides lessons and calculations facilities.

I hope it helps.
Gianmarco
p.s.
I have implemented an Excel's template that calculate some statistics: information available at my site (http://xoomer.virgilio.it/gianmarco.alberti/index_file/Page395.htm) (at the bottom of the page).

Please note: it is the old version; I am working to a new improved version, that is about to be released.
 
#3
Hi,
Thanks for the detailed reply. The normality assumption should be valid for each group individually or should I mix the two groups data and then run the test?
Best regards
 

gianmarco

TS Contributor
#4
Hi,
I believe that normality should be tested for each individual sample.
I think that the smaller sample you have poses a serious issue in order to t-test be performed.

I suggest you to use non-parametric test like Mann-Whitney.

I have just remembered that you can use the permutation t-test, that tests for equality of means using the t statistic, but is non-parametric test with few assumptions. The power of the test is limited by the sample size – significance at the p<0.05 level can only be achieved for n>3 in each sample.

It is provided by a free program (http://folk.uio.no/ohammer/past/) that I consider a true jewel, since it provides many statistical analyses. Many of them are conceived for palaeontologist, but are useful to other needs as well.

I hope it helps,
Gm
 

CB

Super Moderator
#5
As gianmarco says, the choice between parametric and non-parametric tests is more to do with distributional/data characteristics than sample size. I.e., using non-parametric tests is not going to help the fact that your sample size is too small :(

E.g., using the t-test and assuming normality, even if the population difference between the two groups is a full population standard deviation (effect size = 1, a very large population difference), your statistical power or chance of correctly rejecting the null hypothesis is only about 58% (from http://www.dssresearch.com/toolkit/spcalc/power.asp). You usually want power of over 80%. If normality is not present (whether or not you use a non-parametric test), this power will be even lower. Try using the calculator with better estimates of the difference and standard deviations to see how it looks.

In sum: I'd suggest either trying to increase your sample size (especially in the first group), or if this isn't feasible, focusing on descriptive rather than inferential statistics.

Good luck :)