+ Reply to Thread
Results 1 to 3 of 3

Thread: False Coverage Rate - adjusting confidence intervals for multiple comparisons

  1. #1
    Points: 12, Level: 1
    Level completed: 23%, Points required for next Level: 38

    Posts
    2
    Thanks
    1
    Thanked 0 Times in 0 Posts

    False Coverage Rate - adjusting confidence intervals for multiple comparisons




    Hello, I could do with some help regarding correcting confidence intervals for multiple comparisons.

    I have carried out a number of multiple regression analyses, which has given me various different p values. I have used the Benjamini Hochberg procedure to determine which p values survive correction for multiple comparisons.

    However, as my data are not normally distributed, I have used bootstrapping to produce confidence intervals for each p value.

    My first question is - is this necessary? As I understand it, it is the residuals which should be normally distributed in regression, not the IV(s) or DV. Does this mean that I do not need to compute confidence intervals, despite the non-normality of my data? (My residuals are generally normally distributed).

    My second question is - if I do need to compute confidence intervals, I assume that the confidence intervals need to be corrected for multiple comparisons? I have come across a paper by Benjamini and Yekutieli (2005) describing a way to do this, but I am not familiar enough with statistical analysis to understand how to apply the procedure to my own data. So I am looking for an explanation of how to carry out this procedure in language that can be understood by someone who is not very familiar with stats. There is a helpful explanation of the Benjamini-Hochberg correction for p values in here http://www.biostathandbook.com/multiplecomparisons.html - I'm looking for something along those lines, but for confidence intervals instead of p values.

    Thanks in advance!

    Benjamini & Yekutieli (2005). False Discovery Rate–Adjusted Multiple Confidence*Intervals for Selected Parameters, Journal of the American Statistical Association.

  2. #2
    Omega Contributor
    Points: 38,253, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,989
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: False Coverage Rate - adjusting confidence intervals for multiple comparisons

    Yes, I typically correct CIs. Though, I use bonferroni, so I multiply p-value by number of tests; then I divide alpha used in CI by the number of tests.


    So however your correct method works, just replicate with alpha used in the CI calculation.
    Stop cowardice, ban guns!

  3. The Following User Says Thank You to hlsmith For This Useful Post:

    Pepper_Plant (03-03-2016)

  4. #3
    Points: 12, Level: 1
    Level completed: 23%, Points required for next Level: 38

    Posts
    2
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Re: False Coverage Rate - adjusting confidence intervals for multiple comparisons


    Thanks for your reply!

    Do you (or anyone else) know of anywhere that explains the Benjamini and Yekutieli correction procedure for confidence intervals?

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats