+ Reply to Thread
Results 1 to 3 of 3

Thread: 95% CI is Insignificant, P-Value is Significant

  1. #1
    Points: 1,459, Level: 21
    Level completed: 59%, Points required for next Level: 41

    Thanked 0 Times in 0 Posts

    95% CI is Insignificant, P-Value is Significant

    Suppose you're performing a 2x2 contingency table analysis. You do your Chi Square or Fisher Exact test, as appropriate, then you calculate an odds ratio and get your 95% CI for the OR. Under what circumstances can the P-Value for the ChiSquare and Fisher test be significant (e.g. P < 0.05) but the 95% CI of the OR straddles 1.0?

    Is this ever possible using standard statistics techniques?

    What about if you're performing complex survey analysis where the survey is stratified, clustered and weighted? I ask because I've been using SAS to process data from a complex survey design and I've run into a situation where the P-Value and 95% CI conflict:

    Sample Size = 3,618

    Rao-Scott Chi-Square Test
    Pearson Chi-Square 4.1614
    Design Correction 1.0794
    Rao-Scott Chi-Square 3.8551
    DF 1
    Pr > ChiSq 0.0496 *** significant
    F Value 3.8551
    Num DF 1
    Den DF 1765
    Pr > F 0.0498 *** significant

    Odds Ratio (Row1/Row2)
    Estimate 95% Confidence Limits
    Odds Ratio 0.1548 (0.0176-1.3595) *** not significant

  2. #2
    TS Contributor
    Points: 4,330, Level: 41
    Level completed: 90%, Points required for next Level: 20

    Thanked 6 Times in 6 Posts
    Yes this is possible.

    It occurs because the chi square (which is just the square of the asymptotic z-test for proportions) cannot be inverted to give the confidence interval for OR that you are using.

    In general the deal with p-values and CI that you are thinking of only holds if the hypotheis test and CI are connected by a tautoulogy, excuse spelling.


    CI {estimate -se p zquant < parameter < estimate + se* zquant}

    cang be reexpressed with the test stat in the middle of the inequality

    I would be interested to see exact conditions myself, but that is going to be tedious i think

  3. #3
    Devorador de queso
    Points: 95,995, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Tampa, FL
    Thanked 2,630 Times in 2,246 Posts

    Essentially what fed1 is saying is that the program is using one method to compute the p-value and it's using a different method to compute the confidence interval and there is no guarantee those methods will match up exactly. Asymptotically if they're both good methods they should give similar results but we can't always invert a hypothesis test to get a confidence interval very easily but there might be an easier method that works asymptotically...

+ Reply to Thread


Similar Threads

  1. Replies: 4
    Last Post: 08-20-2012, 10:02 AM
  2. Replies: 2
    Last Post: 08-20-2012, 03:26 AM
  3. insignificant variables- what to do?
    By toni_azoulay in forum Regression Analysis
    Replies: 2
    Last Post: 12-04-2010, 01:23 AM
  4. Significant ANOVA but insignificant post hocs
    By psychgrl778 in forum Statistics
    Replies: 2
    Last Post: 01-20-2010, 09:15 PM
  5. Replies: 0
    Last Post: 12-09-2009, 02:36 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Advertise on Talk Stats