+ Reply to Thread
Results 1 to 4 of 4

Thread: 0.05 or 0.01 level of p for multiple comparisons?

  1. #1
    Points: 276, Level: 5
    Level completed: 52%, Points required for next Level: 24

    Location
    Rome, Italy
    Posts
    8
    Thanks
    0
    Thanked 0 Times in 0 Posts

    0.05 or 0.01 level of p for multiple comparisons?




    I wrote a scientific paper measuring ovarian volumes after two different surgical procedures performed on either ovary.
    The measurements were at three different time-points (1, 3 and 6 months). Evaluated patient numbers were different at the three time-poimts since some of them got pregnant and ovarian volumes would not be accurate anymore. I used a paired t-test at each timepoint.
    No significant difference was present at the first two timepoints, whereas at the third (with a smaller numebr of patients, 40 instead of the initial 51) p was 0.04.
    The referee questions that I should use a 0.01 instead of a 0.05:
    "with many comparisons the effect on the type I error rate is to dilute it, hence it is common to consider 0.01 instead of 0.05". May I answer that I agree, however that we chose to stick with the 0.05 threshold since it is not correct to change methods AFTER the results have been analyzed, and we only add a call for caution for a possible type I error in the Discussion?
    Thank you
    Last edited by ludomuzii; 06-28-2015 at 06:11 AM. Reason: typo

  2. #2
    TS Contributor
    Points: 12,227, Level: 72
    Level completed: 45%, Points required for next Level: 223
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,470
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: 0.05 or 0.01 level of p for multiple comparisons?

    Hi, as you only have 3 time points you could run a one way anova with the time points as the factor and finally a post-hoc test. A post hoc test would automaticaly take care of the effect your referee mentioned and there are several to chose from.

    Regards

  3. #3
    Omega Contributor
    Points: 38,289, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,992
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: 0.05 or 0.01 level of p for multiple comparisons?

    Either approaches are fine, you can change alpha or clearly state the limitation of your analytic approach. Many may cavorted the former approach in that there is lalways a risk that readers may not read the discussion.
    Stop cowardice, ban guns!

  4. #4
    Points: 276, Level: 5
    Level completed: 52%, Points required for next Level: 24

    Location
    Rome, Italy
    Posts
    8
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: 0.05 or 0.01 level of p for multiple comparisons?


    Quote Originally Posted by hlsmith View Post
    Either approaches are fine, you can change alpha or clearly state the limitation of your analytic approach. Many may cavorted the former approach in that there is lalways a risk that readers may not read the discussion.

    Thank you very much to both of you

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats