# Thread: Tukey HSD after ANOVA

1. ## Tukey HSD after ANOVA

Hi there,

i have some trouble in understanding the Tukey HSD. Can you help me out please?
I already did a 3x2(low vs middle vs high HRV level x male vs female) ANOVA for reaction time. Due to the ANOVA results I have an interaction between HRV and gender (p> .05).

Now I want to find out which means differ and used TukeyHSD in R. But now I cannot find any significant differences anymore. Have I done something wrong?

This is what i got:

2. ## Re: Tukey HSD after ANOVA

The proof is in the pudding. I would guess the Tukey HSD component is your reason, Honesty. The Tukey HSD has a errorwise correction for multiple tests. The ANOVA tests at 0.05, but the Tukey corrects the 0.05 to take into account multiple tests and adjusts the p-values accordingly. I would guess that if you ran your tests just performing all of the combinations of possible t-tests based on 0.05 level of significance, at the least, the 3 male vs. 3 female would come up significant and would be the reason the ANOVA omnibus test is significant. Though after adjusting for multiple tests this significance gets lost when trying to conservatively mitigate threats of chance.

3. ## Re: Tukey HSD after ANOVA

I'll make two quick points (i) if you have a significant interaction, then why not use Tests of Simple Effects, and (ii) Reaction Time is not Normally Distributed.

4. ## Re: Tukey HSD after ANOVA

So my whole analysis is useless? I tested the reaction time with shapiro test now and its not normally distributed. But my professor still told me to do ANOVA because I have a big sample of n=150.

Besides the fact that i shouldnt use anova, can i jut split my sample into the three hrv groups and do anova separately?

I'm a little confuse now and dont know what to do.. :S

5. ## Re: Tukey HSD after ANOVA

Originally Posted by Flowerpower
So my whole analysis is useless?
I am not saying that you're whole analysis is useless. Rather, if your study is related to a dissertation and/or a article for publication, then you will most likely have to deal with the issues that I mentioned. And, I like would think that your professor would be aware of these concerns.

6. ## Re: Tukey HSD after ANOVA

You have the data which is the first step. Now you just need to make sure the procedures are appropriate. Test the normality of your residuals (errors), if they are normally distributed use ANOVA if they are not the Friedman Test is the nonparametric alternative to the ANOVA.

7. ## Re: Tukey HSD after ANOVA

Originally Posted by victorxstc
An interesting thing that I have learned here (from Dason) is that you don't need a normal distribution of data for using an ANOVA. It is the error term which should be normally distributed.
victor: The dependent variable is a linear function of the error term.

8. ## Re: Tukey HSD after ANOVA

Sorry, I meant the Kruskal-Wallis not Friedman's test. I got my threads intertwined.

9. ## Re: Tukey HSD after ANOVA

Hi, and welcome

I think P < 0.05 is the significant one. P > 0.05 is nonsignificant.

Besides, it is not only the interaction that should be significant in order to give a significant pairwise Tukey P.

--------------

An interesting thing that I have learned here (from Dason) is that you don't need a normal distribution of data for using an ANOVA. It is the error term which should be normally distributed. But, you should also check for the sphericity for a repeated-measures ANOVA.

But don't worry. If ANOVA was not possible, there are still nonparametric alternatives.

-----------------------------

Originally Posted by Dragan
victor: The dependent variable is a linear function of the error term.
Thanks Dragan Actually I didn't get it. You mean we still need to check the distribution of the sample too? But I remember the discussions which led to the conclusion that this is only the distribution of residuals which matters.

Or you mean in this specific example, this specific dependent variable has such linear relationship with the error term?

10. ## Re: Tukey HSD after ANOVA

Originally Posted by hlsmith
Sorry, I meant the Kruskal-Wallis not Friedman's test. I got my threads intertwined.
I think Friedman is more appropriate, as the data is repeated-measures.

11. ## Re: Tukey HSD after ANOVA

Originally Posted by victorxstc
Thanks Dragan Actually I didn't get it. You mean we still need to check the distribution of the sample too? But I remember the discussions which led to the conclusion that this is only the distribution of residuals which matters.

Or you mean in this specific example, this specific dependent variable has such linear relationship with the error term?

What I am saying is that it's going to provide the same answer..i.e. whether you test the normality assumption on the dependent variable (Y) or the error terms (e). For example, in a one-way ANOVA the linear model is: Y_ij = OverallMean + Treatment_j + error, for person i in treatment group j.

12. ## Re: Tukey HSD after ANOVA

Originally Posted by Dragan
What I am saying is that it's going to provide the same answer..i.e. whether you test the normality assumption on the dependent variable (Y) or the error terms (e). For example, in a one-way ANOVA the linear model is: Y_ij = OverallMean + Treatment_j + error, for person i in treatment group j.
So interesting. But what if both dependent and independent variables have distributions other than normal, but similar to each other? I think (if I understood Dason correctly) in that case, despite the non-normal distribution of dependent variable, we have normal distribution of residual?

13. ## Re: Tukey HSD after ANOVA

Originally Posted by Dragan
What I am saying is that it's going to provide the same answer..i.e. whether you test the normality assumption on the dependent variable (Y) or the error terms (e). For example, in a one-way ANOVA the linear model is: Y_ij = OverallMean + Treatment_j + error, for person i in treatment group j.
Which would be fine if we were checking for multivariate normality or something. But not if we just take the raw response and check for normality.

Code:
``````
> dat <- c(rnorm(50), rnorm(50, 100))
> shapiro.test(dat)

Shapiro-Wilk normality test

data:  dat
W = 0.6562, p-value = 5.749e-14

> res <- residuals(lm(dat ~ gl(2, 50)))
> shapiro.test(res)

Shapiro-Wilk normality test

data:  res
W = 0.9908, p-value = 0.7276``````
We can see that the raw data is NOT normally distributed (it's the mixture of two normals). But the residuals do pass a normality test.

14. ## The Following User Says Thank You to Dason For This Useful Post:

hlsmith (09-13-2012)

15. ## Re: Tukey HSD after ANOVA

Originally Posted by victorxstc
So interesting. But what if both dependent and independent variables have distributions other than normal, but similar to each other? I think (if I understood Dason correctly) in that case, despite the non-normal distribution of dependent variable, we have normal distribution of residual?
You're misinterpreting Dason. See, you can look at ANOVA via Regressioin e.g. Dummy Coded vectors (1's and 0's). When you run the regression and when you perform the tests of normality on the error terms for each group your going to get the same results that you would on the dependent variable Y for each group. The reason is that the difference between the actual Y scores and the errors is simply the mean of groups.

16. ## Re: Tukey HSD after ANOVA

If Y given x, or the residuals, are not normally distributed you can try transformations. If that does not work you can try for example a generalized linear model with gamma distribution (of Y| x).

But before you do that I suggest Flowerpower (interesting name!) tell us something about the study.

What is “HRV”? (Abbreviations!#3@*#)
Reaction time of what?
How many observations do you have in each cell (i.e. combination)? Is it the same so that the design is balanced?
Is the interaction significant or not?
Are the main effects significant?

I suggest Flowerpower pick out the females and show us box plots for each level of “HRV”.