2 samples t-test: normal or not?

#1
Hi.
For a 2 sample t-test, where I'm checking the assumptions, I'm getting contradictory (at least to me) results regarding normality, between histograms Q-Q plots and tables.
Looking at the histograms they seem normal, looking at the plots one of the samples is normal, but looking at the p-values none of them are normal.
All the results I got are attached in this post.
If they are not normal, I should proceed with a non-parametric test, correct? Even if one of the samples is normal?
How is the size of N important here?
Thanks!
 

Englund

TS Contributor
#2
Both samples need to be normal in order for the small sample parametric hypothesis tests to be valid. If, on the other hand, the samples are dependent, then it is the values (residuals) given after taking the difference between the variables for each observation that is of interest. If you have a "large" sample, then you can rely on asymptotic results and forthgo with parametric hypothesis testing. I hope this clarifies things a little bit.
 

noetsi

Fortran must die
#3
many test for normality have weak power so you should reject the null, but don't. The QQ plot is arguably the best way to determine normality
 
#5
How did you get SAS to kick out that figure with the normal line. Can you share the basic code?
Hi. Do you mean the normal curve in the histogram?

PROC UNIVARIATE DATA = … ;
VAR … ;
CLASS … ;
QQPLOT … /NORMAL (MU=EST SIGMA = EST);
HISTOGRAM … /NORMAL;
RUN;
PROC GCHART DATA = … ;
HBAR … /TYPE=MEAN SUMVAR=…
FREQLABEL=‘…’ MEANLABEL=‘…’
ERRORBAR=BARS CLM=95 NOFRAME;
/* CLM – confidence level */
RUN;
 
#6
Both samples need to be normal in order for the small sample parametric hypothesis tests to be valid. If, on the other hand, the samples are dependent, then it is the values (residuals) given after taking the difference between the variables for each observation that is of interest. If you have a "large" sample, then you can rely on asymptotic results and forthgo with parametric hypothesis testing. I hope this clarifies things a little bit.
Thanks. The samples are independent. I'm assuming the samples are not normal and the variances are unequal (you can look at the results attached in this reply -- new example I'm using -- and check if I'm right) and proceeding to a non-parametric test (Wilcoxon).
 

Mean Joe

TS Contributor
#8
Looking at the histograms they seem normal, looking at the plots one of the samples is normal, but looking at the p-values none of them are normal.
The histogram for FE college looks good enough, but the histogram for Sixth Form College does not look normal.
Your sample of Sixth Form College is not normal.
As for FE college, I would say it looks good enough for a t-test but you may want to explore some more, as the Q-Q plot indicates the tails are off.
 

noetsi

Fortran must die
#9
Normality primarily influences the statistical tests (the p values). But if you have very non-normal data, and especially outliers in the tails, you should ask why this is occuring. Commonly that, like outlier analysis, tells you something important about your data you would miss otherwise.
 
#10
The histogram for FE college looks good enough, but the histogram for Sixth Form College does not look normal.
Your sample of Sixth Form College is not normal.
As for FE college, I would say it looks good enough for a t-test but you may want to explore some more, as the Q-Q plot indicates the tails are off.
Yes, thank you. Everyone is giving me great insights, thank you all!
So, having one non normal sample, it means that I can not compare the two samples' means by a t-test, but through a non-parametric test, right?
 

Mean Joe

TS Contributor
#11
Yes, try a non-parametric test. It should confirm that the means are not equal (mean of FE college = 190, mean of Sixth Form college = 555).
 

Karabiner

TS Contributor
#12
Unfortunately, there are no non-parametric tests for the equality of
means. One can only compare rank sums (Wilcoxon) or medians
(Median test).

What bothers me with regard to the normality assumption: in case of
small samples there's the necessary assumption that both samples are
drawn from normally distributed populations. The problems with tests
of significance regarding normality are quite known (e.g. lack of power
in the small samples case; or lack of indication whether a "significant"
violation of the assumption is serious), but why is it recommended
to use graphical methods? A Q-Q plot can show us whether there is a
marked deviation from normality in the sample, but how do we
know whether this deviation indicates non-normality in the population?
Any references or explanations for this?

With kind regards

K.
 

noetsi

Fortran must die
#13
It makes sense that there is no means test for non-parametrics which don't assume interval data.

I have never seen the issue of population normality addressed. In practice it is impossible to ever know what the population distribution is so I am not sure what value it would be to know this. More generally all the assumptions of statistics pertain, as far as I know, to the sample and are not to a population. I assume that if the sample meets the assumptions of a given method than you can use it for analysis regardless of the population. You are conducting analysis on sample not the population even if ultimately you hope the results pertain to the population. So why would it matter if the assumptions pertain to the population for analysis?

But it is interesting that I have never seen population assumptions such as normality addressed. Having said all this it occurs to me that I run regression for example on populations. And test for assumptions like equal error variance or multicolinearity. Is that not required when you have the population?
 

Englund

TS Contributor
#14
I assume that if the sample meets the assumptions of a given method than you can use it for analysis regardless of the population. You are conducting analysis on sample not the population even if ultimately you hope the results pertain to the population. So why would it matter if the assumptions pertain to the population for analysis?

But it is interesting that I have never seen population assumptions such as normality addressed.
I would like to disagree. It is the population characteristics that is of interest, not the sample characteristics. We only use sample data because that is usually the only thing we have at hand. In the classical linear regression model, for example, we only care whether [math]\varepsilon_j \forall j, j=1,2,...,N[/math] is normal. We only use the estimated residuals to evaluate whether [math]\varepsilon \sim N[/math] seems plausible.

If the sample data itself would be of interest, why would we then perform hypothesis tests? Wouldn't it be enough to just evaluate whether all the sample moments exactly match the moments of a normally distributed variable then?
 

Karabiner

TS Contributor
#15
I have never seen the issue of population normality addressed.
But we deal with the idea of statistically testing normality very often here.
And testing is about making statements about the population.
In practice it is impossible to ever know what the population distribution is so I am not sure what value it would be to know this.
What we really or ultimately are interested in, are the sampling distributions of the
test statistics, in order to perform the statistical tests. So, usually we not only are
uninterested in the data distribution within the sample, but we even are uninterested
in normality of the data distribution within the population. But AFAIK in case of a small
sample we need the assumption of normality in the population from which the sample
was drawn, in order to make correct statements about the sampling distribution of the
calculated test statistic. With larger samples, the central limit theorem applies.

Now, because the distribution within the sample is not of concern, or only to the
degree it can be used to infer statements about the distribution in the population,
my question was, how or why graphical methods, based just on sample data,
can be used to make such inferences.

With kind regards

K.
 
R

rsuri

Guest
#16
You can use the test for normality option under 'explore' tab, plots option >> histograms. You would want no significance anywhere in order for your sample to be of normal distribution. If it is, then use parametric, otherwise use non-parametric.