# Thread: Using the Right Test to Confirm Separation of Confidence Intervals

1. ## Using the Right Test to Confirm Separation of Confidence Intervals

Hello,

I'm working on a research project (not homework) and I could benefit from a quick consult since some of my stats are a bit rusty. I have a series of experiments that qualify as Bernoulli trials, so I am estimating 95% confidence intervals on the success rates assuming the binomial distribution. I have eleven different methods for invoking the test, so I have 11 different success rates across about 6 different scenarios. I have produced the CI plot in the attached figure.

Clearly in the figure, we can see complete separation in confidence intervals for a few of the lines at most of the points. What I need to do is to provide a p-value of how confident I am of these clear separations. I want to compute a p-value for every pair of the 11 methods at each of the 6 scenarios. Many of these will be trivial, but I expect the p-values to be very small for pairs containing one of those top 3 or 4 lines and any of the bottom lines.

My brain says, this couldn't be as easy as a t-test could it? Am I oversimplifying things to say that we are just comparing the means here? If anyone can help me determine how to compute these pairwise p-values with the appropriate test, I'd appreciate the direction.

2. ## Re: Using the Right Test to Confirm Separation of Confidence Intervals

One other thought was concerning using chi-square analysis using the success/failure counts of the underlying bernoulli trials from which I fit the binomial confidence intervals...

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts