Lets say we have two processes and we want to know which has a higher success rate. So we do two sets of trials. process 1 gives 53 out of 606 and process 2 gives 32 out of 595. So

p1 is 0.0538 with a 95% CL of [0.0371,0.0751]

p2 is 0.0875 with a 95% CL of [0.0662,0.1128]

It would seem that p2 has a higher success rate. However, both only represent one measurement of the success rate and in general these experiment are on different samples. To get the actual success rate we would need to perform multiple experiments of this type and then calculate the mean of the p1s and p2s. My question is how do I calculate the error on this mean? Is it calculated from the distribution of p1 and p2 or do I need to incorporate the confidence limit for the binomial distribution of each experiment in some way? I am happy to stay in a regime where we can make a nearly Gaussian approximation like with the numbers given above. Thanks in advance.

p1 is 0.0538 with a 95% CL of [0.0371,0.0751]

p2 is 0.0875 with a 95% CL of [0.0662,0.1128]

It would seem that p2 has a higher success rate. However, both only represent one measurement of the success rate and in general these experiment are on different samples. To get the actual success rate we would need to perform multiple experiments of this type and then calculate the mean of the p1s and p2s. My question is how do I calculate the error on this mean? Is it calculated from the distribution of p1 and p2 or do I need to incorporate the confidence limit for the binomial distribution of each experiment in some way? I am happy to stay in a regime where we can make a nearly Gaussian approximation like with the numbers given above. Thanks in advance.

Last edited: