# Thread: Multiple comparisons correction help

1. ## Multiple comparisons correction help

Hi,

I have already asked a similar question but did not explain my problem in entirety. Any help would be greatly appreciated!

I am analyzing some data to which I think I need to make some kind of multiple comparisons correction. The data comes from an economic game (Dictator game) and is the form of donations made by subjects ranging from \$ 0.00 to \$ 0.50 (can be any \$ 0.01 denomination). Subjects were primed with an experimental image or an control image I am primarily interested in the effect of which image they were shown on their donation.

I did the following tests all on the same donation data;
1) I used a Mann-Whitney U test to test if there was a difference in the amount donated for each image shown, control or experimental prime (Main hypotheseis).
2)I tested If there was a difference in the amount donated by each gender (Mann-Whitney U test)
3)I tested the relationship between age of dictator and donation using Spearmans Rank
4) Income and Education were categorical, eg. one level of Education would be "Some High School" and one level of Income would be "\$0-\$15 000" for each there was 6 levels. I tested donation against the different levels of Income (Kruscall-Wallis) and did the same for Education (Kruscall-Wallis).
5)Although dictators could choose any value to donate they all choose one of 11 amounts. For each of these amounts I tested if they were over represented by dictators who saw one image compared to the other. For each amount I encoded the data as 1 if they donated this amount and 0 for any other amount and ran a fishers exact test for each amount (11 tests in total). The only one that was significant was \$0.50, those that saw one image gave \$0.50 significantly more often than those that saw the other.

I realize I have gone overboard with tests. I am not too sure too which I should apply an adjustment. ie. If I Bonferroni adjust for the kruscall-wallis test do I just divide alpha by 6 as this is how many levels in income and Education or do I need to take into account the Mann-Whitney tests and Spearmans rank. Also, for the Fishers exact tests the data is encoded differently for each test. Therefore do I need to make a correction or not? What sort of correction should I make?

Thanks

2. ## Re: Multiple comparisons correction help

You should probably correct based on the total number of pairwise comparisons from the kruskal. The second analyses seems like it may be different enough, but may need to be corrected itself regardless of the other analysis for its own total number of pairwise comparisons.

3. ## Re: Multiple comparisons correction help

if you performed pairwise comparisons for the Kruskal, then if my mental math is right you made 15 direct comparisions. So that is your magic number. You can also, instead - just multiply all of those p-values by 15, should be the same thing.

In my opinion, based on your description you probably don't need to control for the other tests in when correcting all of the tests. Just correct each test based on its own number of pairwise comparisions.

Also, you should not post twice - it usually leads to multiple people answering problems that may have already been resolved and wastes peoples time if they read the same thing twice. Can be very vexing.

4. ## Re: Multiple comparisons correction help

Thanks, for your help, I have now seen I can edit posts so will do this in the future.

 Tweet