+ Reply to Thread
Results 1 to 6 of 6

Thread: Mixed ANOVA and non-normally distributed data

  1. #1
    Points: 463, Level: 9
    Level completed: 26%, Points required for next Level: 37

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Mixed ANOVA and non-normally distributed data




    Dear all,

    I have an experiment with three groups of subjects (i.e., three conditions) while each group was tested twice (see attachment; each dot represents data of a single subject; dark blue are the means and MSE). Using Lilliefors test I found that data in Group 2, day 2 is not normal. I tried to make a log-transform, but the data still failed to comply the normality assumption. For the analysis of each of two days separately, I am fine to use non-parametric statistics (Kruskal-Wallis test instead of one-way ANOVA and Mann–Whitney U test instead of two-samples t-test). The question is that I need to test the interaction between conditions and days, which is a mixed ANOVA in a parametric universe (within-subject factor is time of the measurement and between subject factor is a condition). What is the a non-parametric solution for that? If there is a way that still somehow can use parametric tests, I would be happy with this solution either.

    Many thanks!
    Vadim
    Attached Images  

  2. #2
    TS Contributor
    Points: 14,811, Level: 78
    Level completed: 91%, Points required for next Level: 39
    Miner's Avatar
    Location
    Greater Milwaukee area
    Posts
    1,171
    Thanks
    34
    Thanked 405 Times in 363 Posts

    Re: Mixed ANOVA and non-normally distributed data

    Normality of the data is not an issue for ANOVA provided the residuals are normal. What did your residuals look like?

  3. #3
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: Mixed ANOVA and non-normally distributed data

    Quote Originally Posted by Miner View Post
    Normality of the data is not an issue for ANOVA provided the residuals are normal. What did your residuals look like?
    This is my catch-cry, but to be fair here the OP is talking about the distribution within a condition, so s/he is already talking about residuals.

    My answer to this, though, would be either:
    1) Do not worry about. It takes a truly disgusting degree of non-normality and a miniscule sample size to actually make a real difference. The central limit theorem means that the sampling distributions of the parameters in this situation will be **** close to normal, even if the residuals aren't quite normal.
    2) Alternatively, if you really want to play it safe, do your inference by bootstrapping rather than conventional analytic confidence intervals or p values. I would wager dollars to doughnuts that you'll get almost exactly the same answer though.

    I would strongly argue against transformations, as this changes the interpretations of your coefficients - doing so is responding to a very minor problem by introducing a much larger one. I actually would also not use rank-based non-parametric statistics either unless you're very clear on what hypotheses these tests are actually testing, and are comfortable that these are the hypotheses you wish to test. Bootstrapping strikes me as preferable because you can basically use the original model, just with a more robust way of delivering inferences.
    Matt aka CB | twitter.com/matthewmatix

  4. #4
    Points: 463, Level: 9
    Level completed: 26%, Points required for next Level: 37

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Mixed ANOVA and non-normally distributed data

    Thank you a lot Matt for the answer! I need to convince not only myself, but also the reviewers...

    Regarding the bootstrapping option: do you mean specifically bootstrapping, and not permutation. Because for calculating p-value for difference between two samples (aka two sample t-test), I think we lump all the data in one vector, randomly split it to two subsets and calculate the difference between means of two subsets. So, I have a surrogate distribution and can use it instead of the normal distribution.

    What is even less clear for me, is what is the procedure for one-way ANOVA and particularly for mixed two-way ANOVA. First, if I follow the idea of two sample permutation t-test, does it mean that I should manually calculate ANOVA steps, including SS-within and SS-between for permutation sets? Second, for the two-way mixed ANOVA, how I generate my artificial subsets of data. There are two different factors, so it might not be too meaningful to create sub-partitions without taking this into consideration. If, eventually, the procedure is bootstrapping, then obviously all this is not an issue because I create the partition based on the data of the original partition - but will this be equivalent to the parametric mixed ANOVA?

  5. #5
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: Mixed ANOVA and non-normally distributed data

    Quote Originally Posted by vadim View Post
    Regarding the bootstrapping option: do you mean specifically bootstrapping, and not permutation.
    Yeah, what you can do is bootstrap to get confidence intervals and use those to determine whether effects are statistically significant, rather than using p values directly. I don't quite know how to set up a permutation test in this scenario, though no doubt it's possible. In terms of satisfying reviewers, there's plenty of literature knocking around covering the topics at hand (ANOVA robustness to non-normality, bootstrapping and permutation, etc.) so it's just a case of linking the reviewers to evidence that shows your method is appropriate.
    Matt aka CB | twitter.com/matthewmatix

  6. #6
    Points: 463, Level: 9
    Level completed: 26%, Points required for next Level: 37

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Mixed ANOVA and non-normally distributed data


    Thanks a lot for the valuable suggestions!

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats