+ Reply to Thread
Results 1 to 2 of 2

Thread: Reporting effect size/post hocs when not significant- rigorous alpha

  1. #1
    Points: 4, Level: 1
    Level completed: 7%, Points required for next Level: 46

    Thanked 0 Times in 0 Posts

    Reporting effect size/post hocs when not significant- rigorous alpha


    I have received my manuscript back from reviewers who have suggested that I report Cohen's d for differences found between three groups. Is it appropriate to report them even when the differences were not significant? My alpha was .005 (after Bonferonni adjustment- I have 11 DVs), so even when the p value is .01, the difference is not significant, but the effect sizes are decent.

    Another question: Can I run post-hocs if my overall ANOVA (actually it was a Welch's F-test to determine differences among three unequal groups) is not significant? The ANOVA is not significant at the .005 level, but some of the post-hocs are.

    Finally, reviewers asked me to additionally build models with dummy coded variables to see if they predict the DVs. if I set an alpha for .005 for my F-test hypotheses, does that mean I need to use a .005 alpha for my regression analyses, too? What about my correlation table of all variables (I would think that I can just use .05 for the correlation table of all variables).

    Thank you for your help!

  2. #2
    Omega Contributor
    Points: 38,253, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Not Ames, IA
    Thanked 1,185 Times in 1,146 Posts

    Re: Reporting effect size/post hocs when not significant- rigorous alpha

    In regards to your first question, yes you can report these effect sizes. It helps the reader judge the magnitude, direction, and potentially power of your test. Just add adjusted confidence intervals which match your adjusted alpha level.
    Stop cowardice, ban guns!

+ Reply to Thread


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Advertise on Talk Stats