+ Reply to Thread
Results 1 to 4 of 4

Thread: Multiple comparisons correction help

  1. #1
    Points: 181, Level: 3
    Level completed: 62%, Points required for next Level: 19

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Multiple comparisons correction help




    Hi,

    I have already asked a similar question but did not explain my problem in entirety. Any help would be greatly appreciated!

    I am analyzing some data to which I think I need to make some kind of multiple comparisons correction. The data comes from an economic game (Dictator game) and is the form of donations made by subjects ranging from $ 0.00 to $ 0.50 (can be any $ 0.01 denomination). Subjects were primed with an experimental image or an control image I am primarily interested in the effect of which image they were shown on their donation.

    I did the following tests all on the same donation data;
    1) I used a Mann-Whitney U test to test if there was a difference in the amount donated for each image shown, control or experimental prime (Main hypotheseis).
    2)I tested If there was a difference in the amount donated by each gender (Mann-Whitney U test)
    3)I tested the relationship between age of dictator and donation using Spearmans Rank
    4) Income and Education were categorical, eg. one level of Education would be "Some High School" and one level of Income would be "$0-$15 000" for each there was 6 levels. I tested donation against the different levels of Income (Kruscall-Wallis) and did the same for Education (Kruscall-Wallis).
    5)Although dictators could choose any value to donate they all choose one of 11 amounts. For each of these amounts I tested if they were over represented by dictators who saw one image compared to the other. For each amount I encoded the data as 1 if they donated this amount and 0 for any other amount and ran a fishers exact test for each amount (11 tests in total). The only one that was significant was $0.50, those that saw one image gave $0.50 significantly more often than those that saw the other.

    I realize I have gone overboard with tests. I am not too sure too which I should apply an adjustment. ie. If I Bonferroni adjust for the kruscall-wallis test do I just divide alpha by 6 as this is how many levels in income and Education or do I need to take into account the Mann-Whitney tests and Spearmans rank. Also, for the Fishers exact tests the data is encoded differently for each test. Therefore do I need to make a correction or not? What sort of correction should I make?

    Thanks
    Last edited by JonoBone; 03-01-2013 at 10:59 AM.

  2. #2
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Multiple comparisons correction help

    You should probably correct based on the total number of pairwise comparisons from the kruskal. The second analyses seems like it may be different enough, but may need to be corrected itself regardless of the other analysis for its own total number of pairwise comparisons.
    Stop cowardice, ban guns!

  3. #3
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Multiple comparisons correction help

    if you performed pairwise comparisons for the Kruskal, then if my mental math is right you made 15 direct comparisions. So that is your magic number. You can also, instead - just multiply all of those p-values by 15, should be the same thing.

    In my opinion, based on your description you probably don't need to control for the other tests in when correcting all of the tests. Just correct each test based on its own number of pairwise comparisions.

    Also, you should not post twice - it usually leads to multiple people answering problems that may have already been resolved and wastes peoples time if they read the same thing twice. Can be very vexing.
    Stop cowardice, ban guns!

  4. #4
    Points: 181, Level: 3
    Level completed: 62%, Points required for next Level: 19

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Multiple comparisons correction help


    Thanks, for your help, I have now seen I can edit posts so will do this in the future.

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats