+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 16

Thread: Tukey HSD after ANOVA

  1. #1
    Points: 2,489, Level: 30
    Level completed: 26%, Points required for next Level: 111

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Tukey HSD after ANOVA




    Hi there,

    i have some trouble in understanding the Tukey HSD. Can you help me out please?
    I already did a 3x2(low vs middle vs high HRV level x male vs female) ANOVA for reaction time. Due to the ANOVA results I have an interaction between HRV and gender (p> .05).

    Now I want to find out which means differ and used TukeyHSD in R. But now I cannot find any significant differences anymore. Have I done something wrong?

    This is what i got:
    Attached Images  

  2. #2
    Omega Contributor
    Points: 38,253, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,989
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: Tukey HSD after ANOVA

    The proof is in the pudding. I would guess the Tukey HSD component is your reason, Honesty. The Tukey HSD has a errorwise correction for multiple tests. The ANOVA tests at 0.05, but the Tukey corrects the 0.05 to take into account multiple tests and adjusts the p-values accordingly. I would guess that if you ran your tests just performing all of the combinations of possible t-tests based on 0.05 level of significance, at the least, the 3 male vs. 3 female would come up significant and would be the reason the ANOVA omnibus test is significant. Though after adjusting for multiple tests this significance gets lost when trying to conservatively mitigate threats of chance.

  3. #3
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: Tukey HSD after ANOVA

    I'll make two quick points (i) if you have a significant interaction, then why not use Tests of Simple Effects, and (ii) Reaction Time is not Normally Distributed.

  4. #4
    Points: 2,489, Level: 30
    Level completed: 26%, Points required for next Level: 111

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Tukey HSD after ANOVA

    So my whole analysis is useless? I tested the reaction time with shapiro test now and its not normally distributed. But my professor still told me to do ANOVA because I have a big sample of n=150.

    Besides the fact that i shouldnt use anova, can i jut split my sample into the three hrv groups and do anova separately?

    I'm a little confuse now and dont know what to do.. :S

  5. #5
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: Tukey HSD after ANOVA

    Quote Originally Posted by Flowerpower View Post
    So my whole analysis is useless?
    I am not saying that you're whole analysis is useless. Rather, if your study is related to a dissertation and/or a article for publication, then you will most likely have to deal with the issues that I mentioned. And, I like would think that your professor would be aware of these concerns.

  6. #6
    Omega Contributor
    Points: 38,253, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,989
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: Tukey HSD after ANOVA

    You have the data which is the first step. Now you just need to make sure the procedures are appropriate. Test the normality of your residuals (errors), if they are normally distributed use ANOVA if they are not the Friedman Test is the nonparametric alternative to the ANOVA.

  7. #7
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: Tukey HSD after ANOVA

    Quote Originally Posted by victorxstc View Post
    An interesting thing that I have learned here (from Dason) is that you don't need a normal distribution of data for using an ANOVA. It is the error term which should be normally distributed.
    victor: The dependent variable is a linear function of the error term.

  8. #8
    Omega Contributor
    Points: 38,253, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,989
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: Tukey HSD after ANOVA

    Sorry, I meant the Kruskal-Wallis not Friedman's test. I got my threads intertwined.

  9. #9
    Pirate
    Points: 15,159, Level: 79
    Level completed: 62%, Points required for next Level: 191
    victorxstc's Avatar
    Posts
    875
    Thanks
    229
    Thanked 332 Times in 297 Posts

    Re: Tukey HSD after ANOVA

    Hi, and welcome

    I think P < 0.05 is the significant one. P > 0.05 is nonsignificant.

    Besides, it is not only the interaction that should be significant in order to give a significant pairwise Tukey P.

    --------------

    An interesting thing that I have learned here (from Dason) is that you don't need a normal distribution of data for using an ANOVA. It is the error term which should be normally distributed. But, you should also check for the sphericity for a repeated-measures ANOVA.

    But don't worry. If ANOVA was not possible, there are still nonparametric alternatives.

    -----------------------------

    Quote Originally Posted by Dragan View Post
    victor: The dependent variable is a linear function of the error term.
    Thanks Dragan Actually I didn't get it. You mean we still need to check the distribution of the sample too? But I remember the discussions which led to the conclusion that this is only the distribution of residuals which matters.

    Or you mean in this specific example, this specific dependent variable has such linear relationship with the error term?

  10. #10
    Pirate
    Points: 15,159, Level: 79
    Level completed: 62%, Points required for next Level: 191
    victorxstc's Avatar
    Posts
    875
    Thanks
    229
    Thanked 332 Times in 297 Posts

    Re: Tukey HSD after ANOVA

    Quote Originally Posted by hlsmith View Post
    Sorry, I meant the Kruskal-Wallis not Friedman's test. I got my threads intertwined.
    I think Friedman is more appropriate, as the data is repeated-measures.

  11. #11
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: Tukey HSD after ANOVA

    Quote Originally Posted by victorxstc View Post
    Thanks Dragan Actually I didn't get it. You mean we still need to check the distribution of the sample too? But I remember the discussions which led to the conclusion that this is only the distribution of residuals which matters.

    Or you mean in this specific example, this specific dependent variable has such linear relationship with the error term?

    What I am saying is that it's going to provide the same answer..i.e. whether you test the normality assumption on the dependent variable (Y) or the error terms (e). For example, in a one-way ANOVA the linear model is: Y_ij = OverallMean + Treatment_j + error, for person i in treatment group j.

  12. #12
    Pirate
    Points: 15,159, Level: 79
    Level completed: 62%, Points required for next Level: 191
    victorxstc's Avatar
    Posts
    875
    Thanks
    229
    Thanked 332 Times in 297 Posts

    Re: Tukey HSD after ANOVA

    Quote Originally Posted by Dragan View Post
    What I am saying is that it's going to provide the same answer..i.e. whether you test the normality assumption on the dependent variable (Y) or the error terms (e). For example, in a one-way ANOVA the linear model is: Y_ij = OverallMean + Treatment_j + error, for person i in treatment group j.
    So interesting. But what if both dependent and independent variables have distributions other than normal, but similar to each other? I think (if I understood Dason correctly) in that case, despite the non-normal distribution of dependent variable, we have normal distribution of residual?

  13. #13
    Devorador de queso
    Points: 95,540, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,930
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: Tukey HSD after ANOVA

    Quote Originally Posted by Dragan View Post
    What I am saying is that it's going to provide the same answer..i.e. whether you test the normality assumption on the dependent variable (Y) or the error terms (e). For example, in a one-way ANOVA the linear model is: Y_ij = OverallMean + Treatment_j + error, for person i in treatment group j.
    Which would be fine if we were checking for multivariate normality or something. But not if we just take the raw response and check for normality.

    Code: 
    
    > dat <- c(rnorm(50), rnorm(50, 100))
    > shapiro.test(dat)
    
    	Shapiro-Wilk normality test
    
    data:  dat 
    W = 0.6562, p-value = 5.749e-14
    
    > res <- residuals(lm(dat ~ gl(2, 50)))
    > shapiro.test(res)
    
    	Shapiro-Wilk normality test
    
    data:  res 
    W = 0.9908, p-value = 0.7276
    We can see that the raw data is NOT normally distributed (it's the mixture of two normals). But the residuals do pass a normality test.
    I don't have emotions and sometimes that makes me very sad.

  14. The Following User Says Thank You to Dason For This Useful Post:

    hlsmith (09-13-2012)

  15. #14
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: Tukey HSD after ANOVA

    Quote Originally Posted by victorxstc View Post
    So interesting. But what if both dependent and independent variables have distributions other than normal, but similar to each other? I think (if I understood Dason correctly) in that case, despite the non-normal distribution of dependent variable, we have normal distribution of residual?
    You're misinterpreting Dason. See, you can look at ANOVA via Regressioin e.g. Dummy Coded vectors (1's and 0's). When you run the regression and when you perform the tests of normality on the error terms for each group your going to get the same results that you would on the dependent variable Y for each group. The reason is that the difference between the actual Y scores and the errors is simply the mean of groups.

  16. #15
    Human
    Points: 12,666, Level: 73
    Level completed: 54%, Points required for next Level: 184
    Awards:
    Master Tagger
    GretaGarbo's Avatar
    Posts
    1,360
    Thanks
    455
    Thanked 462 Times in 402 Posts

    Re: Tukey HSD after ANOVA


    If Y given x, or the residuals, are not normally distributed you can try transformations. If that does not work you can try for example a generalized linear model with gamma distribution (of Y| x).

    But before you do that I suggest Flowerpower (interesting name!) tell us something about the study.

    What is “HRV”? (Abbreviations!#3@*#)
    Reaction time of what?
    How many observations do you have in each cell (i.e. combination)? Is it the same so that the design is balanced?
    Is the interaction significant or not?
    Are the main effects significant?

    I suggest Flowerpower pick out the females and show us box plots for each level of “HRV”.

+ Reply to Thread
Page 1 of 2 1 2 LastLast

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats