+ Reply to Thread
Results 1 to 4 of 4

Thread: Confidence intervals for proportions: approximating a discrete distribution with a co

  1. #1
    ggplot2orBust
    Points: 71,220, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    User with most referrers
    trinker's Avatar
    Location
    Buffalo, NY
    Posts
    4,417
    Thanks
    1,811
    Thanked 928 Times in 809 Posts

    Confidence intervals for proportions: approximating a discrete distribution with a co




    I saw on this website http://onlinestatbook.com/2/estimati...ortion_ci.html

    The following quote about calculating a CI for a proportion:

    To correct for the fact that we are approximating a discrete distribution with a continuous distribution (the normal distribution), we subtract 0.5/N from the lower limit and add 0.5/N to the upper limit of the interval.
    Giving: p \pm  Z_{.95}\sqrt{\frac{p(1-p)}{N}} \pm  \frac{.5}{N}

    Where (it appears) N is the sample size.

    Another website:

    http://stattrek.com/estimation/confi...px?Tutorial=AP

    has the approximation simply as:

    p \pm  Z_{.95}\sqrt{\frac{p(1-p)}{n}}

    Where n is the sample size.

    The first approach can give values larger than 1 or smaller than 0. Here the vector has all 1s though it was possible to have gotten a zero. The result with te first formula above gives the following CI:

    Code: 
    set.seed(10)
    x <- sample(0:1, 100, TRUE, c(.001, .999))
    
    [0.995, 1.005]
    This seems bad (> 1). Do we indeed need to:

    correct for the fact that we are approximating a discrete distribution with a continuous distribution
    With...

    \pm  \frac{.5}{N}
    "If you torture the data long enough it will eventually confess."
    -Ronald Harry Coase -

  2. #2
    TS Contributor
    Points: 7,081, Level: 55
    Level completed: 66%, Points required for next Level: 69

    Location
    Copenhagen , Denmark
    Posts
    515
    Thanks
    71
    Thanked 123 Times in 116 Posts

    Re: Confidence intervals for proportions: approximating a discrete distribution with

    In this instance you have specifically chosen probabilities p=0.001 that are extreme compared to sample size 100. The expected number of succeses is ceiling(99.9)=100. When approximating a binomial the poisson dist. is better for "very small" p and the normal will work best for p around 0.5. The way I see it you are choosing p=0.001 thereby creating a problem for the normal distribution approximation that more relates to the problem of approximating a bounded distribution taking values in 0,1,..,100 with an unbounded -inf,inf than it relates to the problem of approximating something discrete with a continuous distribution. The continuity correction does not solve the first type of these problems and the fact that the problem exists is no news.

    So to answer the question I guess you could do a simulation study on samples where you in the first place actually would use a normal distribution or where the problem of approximation is a problem relating to what the correction is intended to correct.

    As an alternative you could simple invent the Trinker continuity approximation where in the case you get a result below 0 you round up to zero and in the case you get a value above 1 you round down

  3. The Following User Says Thank You to JesperHP For This Useful Post:

    trinker (02-16-2016)

  4. #3
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Confidence intervals for proportions: approximating a discrete distribution with

    I believe I recall seeing this correction before, but I think Jesper hit the nail on the head.
    Stop cowardice, ban guns!

  5. #4
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Confidence intervals for proportions: approximating a discrete distribution with


    Not sure if this would be of interest:


    http://www.r-bloggers.com/confidence...r-proportions/
    Stop cowardice, ban guns!

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats