How to present data after Holm-Bonferroni Correction

[artery]

Hi all,

I am manually using Holm-Bonferroni method to correct my data as I can't find a way to do it using software. As the data is relatively small, it is still manageable. The problem is how I should present the data after the correction.

1) When the null hypothesis is not rejected as the P-value is <0.05 but more than the adjusted α, should I indicate that anywhere or it is alright to just write it off as not significant?

2) When the null hypothesis is rejected as the P-value is still less than the adjusted α, do the P-value and the *** that represent the significance level still remain the same as before correction?

Thank you.

 

[Asymptotically Unbiased]

Hi Artery,

If you have a list of p-values that you want to correct for multiple comparisons, it's very easy to do so using R, which is free: just visit CRAN, choose your operating system, and install.

P-value adjustment is done using the p.adjust command: simply give it a vector of p-values, and it will spit out the adjusted values. This is easier than doing things by hand, and it's also perfectly reproducible.

Here's some code you can use as an example: You can copy and paste the blue text into R, and modify it to your needs.
# Hashes denote comments in R - these won't be read by the compiler.
# First create a vector of your unadjusted p-values:
# just replace these values with your p-values
unadjusted <- c(0.026, 0.4772, 0.2029, 0.7272, 0.0325,
0.0308, 0.005, 0.7094, 0.2216, 0.185)

# Next, call the p.adjust function
adjusted <- p.adjust(unadjusted, method = "holm")

# Now output these side by side, so you can compare
p.table <- cbind(adjusted, unadjusted)
p.table

This should give you the following:
adjusted unadjusted
[1,] 0.2340 0.0260
[2,] 1.0000 0.4772
[3,] 1.0000 0.2029
[4,] 1.0000 0.7272
[5,] 0.2464 0.0325
[6,] 0.2464 0.0308
[7,] 0.0500 0.0050
[8,] 1.0000 0.7094
[9,] 1.0000 0.2216
[10,] 1.0000 0.1850

It's probably easiest just to report the adjusted values, but I have seen the unadjusted values annotated to denote changes in statistical significance. I prefer the former, and will let Andrew Gelman and Hal Stern explain why, as they'll do a much better job than I would.

HTH