+ Reply to Thread
Results 1 to 5 of 5

Thread: Overlapping confidence intervals

  1. #1
    Points: 1,801, Level: 25
    Level completed: 1%, Points required for next Level: 99

    Posts
    11
    Thanks
    7
    Thanked 0 Times in 0 Posts

    Overlapping confidence intervals




    I have run an experiment looking at how repeatable urban and rural birds are in response to a stimulus (i.e. do they behave in the same way each time or change their behaviours). I have tested this using the rptR pacakage in R and found a significant result for rural birds (they are consistent) but not for urban birds (they are not consistent).

    These results are interesting. However, the 95% confidence intervals for the two groups overlap which I think suggests no significant difference between groups.

    So my question is, how do I interpret overlapping confidence intervals when one group is significant and the other is not? I can't say there is a statistical difference between populations, but one population shows significant repeatability and the other doesn't.

    The attached graph shows my results.

    Thanks!
    Attached Images  

  2. #2
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Overlapping confidence intervals

    So the whiskers represent 95% confidence intervals? It sounds like you are blending to procedures together. How is repeatability defined to get a significant results?


    So would this be like you testing that a value is not equal to zero, so you slap 95% CI on it, then you use that interval to compare it to another group? I think you are probably better off just actually testing your secondary hypothesis. Does rptR stand for Raptor?
    Stop cowardice, ban guns!

  3. The Following User Says Thank You to hlsmith For This Useful Post:

    shardman (10-11-2016)

  4. #3
    TS Contributor
    Points: 14,811, Level: 78
    Level completed: 91%, Points required for next Level: 39
    Miner's Avatar
    Location
    Greater Milwaukee area
    Posts
    1,171
    Thanks
    34
    Thanked 405 Times in 363 Posts

    Re: Overlapping confidence intervals

    This is a common misunderstanding. Confidence interval may overlap up to 25-30% and still be significant. See the linked article from Cornell University entitled Overlapping Confidence Intervals and Statistical Significance.

  5. The Following User Says Thank You to Miner For This Useful Post:

    shardman (10-11-2016)

  6. #4
    Points: 1,801, Level: 25
    Level completed: 1%, Points required for next Level: 99

    Posts
    11
    Thanks
    7
    Thanked 0 Times in 0 Posts

    Re: Overlapping confidence intervals

    Hi hlsmith,

    Thanks for your answer. rptR is the name of a package for measuring repeatability using R. Repeatability is defined as

    between individual variance / between individual variance + residual variance

    This gives a score from 0 to 1 where 1 means that individuals always respond in exactly the same way when they are measured and 0 means no consistency at all in responses. If the result includes 0 then it is non-significant. This is essentially the same as intra-class correlation (ICC) tests. rptR extracts these variances from GLMM models.

    The plot in my post shows two separate tests one for urban and one for rural birds. Whether it even makes sense to compare confidence intervals from independent tests I don't know. The behaviour measured is the same in urban and rural birds but this is not directly a test to see if urban and rural are different from each other, just to see if either group shows significant repeatability. What I'm not sure of now is if I can say that urban and rural birds are different because rural birds show significant repeatability (P =<0.0001) and urban birds don't.

  7. #5
    Points: 1,801, Level: 25
    Level completed: 1%, Points required for next Level: 99

    Posts
    11
    Thanks
    7
    Thanked 0 Times in 0 Posts

    Re: Overlapping confidence intervals


    Thanks for the link Miner, that's interesting!

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats