Mixed ANOVA and non-normally distributed data

vadim

New Member
#1
Dear all,

I have an experiment with three groups of subjects (i.e., three conditions) while each group was tested twice (see attachment; each dot represents data of a single subject; dark blue are the means and MSE). Using Lilliefors test I found that data in Group 2, day 2 is not normal. I tried to make a log-transform, but the data still failed to comply the normality assumption. For the analysis of each of two days separately, I am fine to use non-parametric statistics (Kruskal-Wallis test instead of one-way ANOVA and Mann–Whitney U test instead of two-samples t-test). The question is that I need to test the interaction between conditions and days, which is a mixed ANOVA in a parametric universe (within-subject factor is time of the measurement and between subject factor is a condition). What is the a non-parametric solution for that? If there is a way that still somehow can use parametric tests, I would be happy with this solution either.

Many thanks!
Vadim
 

Miner

TS Contributor
#2
Normality of the data is not an issue for ANOVA provided the residuals are normal. What did your residuals look like?
 

CowboyBear

Super Moderator
#3
Normality of the data is not an issue for ANOVA provided the residuals are normal. What did your residuals look like?
This is my catch-cry, but to be fair here the OP is talking about the distribution within a condition, so s/he is already talking about residuals.

My answer to this, though, would be either:
1) Do not worry about. It takes a truly disgusting degree of non-normality and a miniscule sample size to actually make a real difference. The central limit theorem means that the sampling distributions of the parameters in this situation will be **** close to normal, even if the residuals aren't quite normal.
2) Alternatively, if you really want to play it safe, do your inference by bootstrapping rather than conventional analytic confidence intervals or p values. I would wager dollars to doughnuts that you'll get almost exactly the same answer though.

I would strongly argue against transformations, as this changes the interpretations of your coefficients - doing so is responding to a very minor problem by introducing a much larger one. I actually would also not use rank-based non-parametric statistics either unless you're very clear on what hypotheses these tests are actually testing, and are comfortable that these are the hypotheses you wish to test. Bootstrapping strikes me as preferable because you can basically use the original model, just with a more robust way of delivering inferences.
 

vadim

New Member
#4
Thank you a lot Matt for the answer! I need to convince not only myself, but also the reviewers...

Regarding the bootstrapping option: do you mean specifically bootstrapping, and not permutation. Because for calculating p-value for difference between two samples (aka two sample t-test), I think we lump all the data in one vector, randomly split it to two subsets and calculate the difference between means of two subsets. So, I have a surrogate distribution and can use it instead of the normal distribution.

What is even less clear for me, is what is the procedure for one-way ANOVA and particularly for mixed two-way ANOVA. First, if I follow the idea of two sample permutation t-test, does it mean that I should manually calculate ANOVA steps, including SS-within and SS-between for permutation sets? Second, for the two-way mixed ANOVA, how I generate my artificial subsets of data. There are two different factors, so it might not be too meaningful to create sub-partitions without taking this into consideration. If, eventually, the procedure is bootstrapping, then obviously all this is not an issue because I create the partition based on the data of the original partition - but will this be equivalent to the parametric mixed ANOVA?
 

CowboyBear

Super Moderator
#5
Regarding the bootstrapping option: do you mean specifically bootstrapping, and not permutation.
Yeah, what you can do is bootstrap to get confidence intervals and use those to determine whether effects are statistically significant, rather than using p values directly. I don't quite know how to set up a permutation test in this scenario, though no doubt it's possible. In terms of satisfying reviewers, there's plenty of literature knocking around covering the topics at hand (ANOVA robustness to non-normality, bootstrapping and permutation, etc.) so it's just a case of linking the reviewers to evidence that shows your method is appropriate.