+ Reply to Thread
Results 1 to 2 of 2

Thread: Multiple studies all showing p < 0.05 (NOT asking about multiple testing)

  1. #1
    Points: 4, Level: 1
    Level completed: 7%, Points required for next Level: 46

    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Multiple studies all showing p < 0.05 (NOT asking about multiple testing)




    Intuitively, we are more confident in a result if multiple methods or research groups all show the same thing.

    So my question is: if 5 independent people do the same experiment, and they each replicate it enough that they each independently get p<0.05 in rejecting the null hypothesis, what can we say about a "p-value" in aggregate?

    Can we multiply their p-values? As in 0.01 * 0.04 * 0.03 * 0.001 * 0.03? Or is there a different procedure?

    Or do we need to add up their raw data and then calculate a new p-value? Or would that give the same result as the above?

    Or does this question not even make sense from the point of view of calculating p-values?

  2. #2
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Multiple studies all showing p < 0.05 (NOT asking about multiple testing)


    I have never seen p values of multiple studies done this way. If multiple studies come to similar findings that provides support for the result, but not a way to average p values. It is possible that you might find something in meta analysis to help with such conclusions.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats