Exact p-value conflict with exact 95% confidence interval

#1
Dear all,

How would you interpret a situation where Pearson's exact p-value is 0.01, but the exact confidence interval for the risk ratio embraces 1, such as in the example below?

Exposed and sick: 25
Exposed and well: 19
Unexposed and sick: 1
Unexposed and well: 9

Exact confidence interval (calculated in Stata 11.1 with the cs command): 5.68, 95% confidence interval 0.87 to 37.1.

Fisher's exact 2-sided p-value: 0.012. (Apologies for prior error)

Sorry to bother you with such a triviality!

Patrick
 
Last edited:

Link

Ninja say what!?!
#2
I think your p-value is wrong.
\( RR=\frac{P(D^{+}|E^{+})}{P(D^{+}|E^{-})}=5.68 \)
\( CI_{95}=ln(RR) \pm 1.96(SE_{ln(RR)}) \)
\( SE_{ln(RR)}=\sqrt{\frac{1}{D^{+},E^{+}}-\frac{1}{E^{+}}+\frac{1}{D^{+},E^{-}}-\frac{1}{E^{-}}}=0.9577 \)
\( CI_{95}=(0.87, 37.1) \)

Since the ln(RR) assumes a normal distribution, you get the z statistic and calculate the p-value from it.

\( z = \frac{ln(RR)-0}{SE} = \frac{ln(5.68)}{0.9577}=1.81 \)
\( p value = 0.0703 \)

PS. These formulas take a lot of work to write up! Props go to those who go through the effort to write them!
 
Last edited:
#3
Your p-value is correct. But it is Fisher's Exact. This is a slightly different test. Sometimes used interchangably with other measures. Sometimes gives similar results, sometimes not. It just so happens to be more useful when samples are small - like yours. Because your sample is small, your CI is large, and therefore captures 1. The Pearson value that you may be looking at (close to .0075) is the chi-square test of independence.

Hope that helps.
 

fed1

TS Contributor
#4
Aside from the sample size issues mentioned by lum hearts, the test based on the Relative risk is not the same as that based the risk difference.

The test based on the risk difference is chi square.

Both are standard normal but are apparently not equal.

I say this because link has the test based on ln(RR) as
1.81

wheras the sqrt( chisq ) = 2.67, about

I dont think they are asymptotically equal either, though I may be wrong on that point.
 

Link

Ninja say what!?!
#5
Hi Fed,

Would you mind expanding on your explanation? I'm having a hard time understanding what you're saying.
 

fed1

TS Contributor
#6
I was thinking that because

z-test for proportions = sqrt( chisquare ) = 2.67

is the most powerful test based on the risk difference, asymptotically

and the test based on the log(RR) = 1.81

is the most powerful test based on the ln(RR), asymptotically


They have the same distribution, so they must have different p-values.

This implies that the two tests are not equivalent, that is there are data sets for which the first test rejects but the latter does not?
 
#7
Thank you for your help - I'm going to make the judgement of whether there is an association or not on the basis of the Fisher's exact test result, and warn the readers to interpret the Z-test derived confidence interval with a pinch of salt (for reasons of small samples).