Multiple-testing correction in Moderation analyses

#1
Hi Everyone!

In an exploratory attempt, I ran 35 moderation analyses (both 2-way and 3-way interactions) of which 7 were significant. NOw the reviewer is warning me about false positives and prompting to run some statistical check that would help to identify those. I tried the conservative Bonferroni and the more liberal Benjamini-Hochberg procedures and both of them literally kill my results.

After reading zillions of threads, papers and entries about multiple-testing, i came to realize that none of them discussed moderation analysis, i.e. they talked about the predictive potential of single variables rather than interaction terms.

To what extent might multiple testing be a problem in moderation analysis? I intuitively think that moderation analyses might be more robust to the alpha inflation due to multiple testing, but I guess I cannot quote my intuition in a manuscript. ;)

Any thoughts in this regard?

Thank you in advance! :)
 
Last edited: