False Coverage Rate - adjusting confidence intervals for multiple comparisons

#1
Hello, I could do with some help regarding correcting confidence intervals for multiple comparisons.

I have carried out a number of multiple regression analyses, which has given me various different p values. I have used the Benjamini Hochberg procedure to determine which p values survive correction for multiple comparisons.

However, as my data are not normally distributed, I have used bootstrapping to produce confidence intervals for each p value.

My first question is - is this necessary? As I understand it, it is the residuals which should be normally distributed in regression, not the IV(s) or DV. Does this mean that I do not need to compute confidence intervals, despite the non-normality of my data? (My residuals are generally normally distributed).

My second question is - if I do need to compute confidence intervals, I assume that the confidence intervals need to be corrected for multiple comparisons? I have come across a paper by Benjamini and Yekutieli (2005) describing a way to do this, but I am not familiar enough with statistical analysis to understand how to apply the procedure to my own data. So I am looking for an explanation of how to carry out this procedure in language that can be understood by someone who is not very familiar with stats. There is a helpful explanation of the Benjamini-Hochberg correction for p values in here http://www.biostathandbook.com/multiplecomparisons.html - I'm looking for something along those lines, but for confidence intervals instead of p values.

Thanks in advance!

Benjamini & Yekutieli (2005). False Discovery Rate–Adjusted Multiple Confidence*Intervals for Selected Parameters, Journal of the American Statistical Association.
 

hlsmith

Omega Contributor
#2
Yes, I typically correct CIs. Though, I use bonferroni, so I multiply p-value by number of tests; then I divide alpha used in CI by the number of tests.


So however your correct method works, just replicate with alpha used in the CI calculation.
 
#3
Thanks for your reply!

Do you (or anyone else) know of anywhere that explains the Benjamini and Yekutieli correction procedure for confidence intervals?