+ Reply to Thread
Results 1 to 5 of 5

Thread: Help - is a collection of studies normally distributed if the n >150? false positives

  1. #1
    Points: 7, Level: 1
    Level completed: 13%, Points required for next Level: 43

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Smile Help - is a collection of studies normally distributed if the n >150? false positives




    Hi,

    I have a collection of medical trials, all for different conditions and different medications. There are 160 in all. They have all different methodologies etc. They come up with a result of either "positive" drug worked or "negative" no effect of drug.

    I want to estimate the number of false positives.....

    Can I say that the number of false positives is 1/20 (5%) - by assuming the trials are normally distributed as n is large?

    Thx

    Liz

  2. #2
    TS Contributor
    Points: 17,779, Level: 84
    Level completed: 86%, Points required for next Level: 71
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,542
    Thanks
    56
    Thanked 640 Times in 602 Posts

    Re: Help - is a collection of studies normally distributed if the n >150? false posit

    No variable becomes normally distributed just because sample size is large.

    You could asses the probability of a false rejection in each trial, using
    its actual p-value, and combine these probabilities across trial. But
    that's just theory. Here you can read what really is the case.

    With kind regards

    K.

  3. #3
    Points: 7, Level: 1
    Level completed: 13%, Points required for next Level: 43

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Help - is a collection of studies normally distributed if the n >150? false posit

    Many Thanks K!!!

    So, can I just confirm - for this meta-analysis of 160 different trials testing the efficacy of a particular "style" of treatment - its wrong to say 8/160 will be "positive" due to chance alone - the only way to assess probability of their being "false positives" is to analyse this in each trial and somehow gather together the results (apologies for repeating myself, I find this a bit tricky)

    Many Thanks


  4. #4
    TS Contributor
    Points: 17,779, Level: 84
    Level completed: 86%, Points required for next Level: 71
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,542
    Thanks
    56
    Thanked 640 Times in 602 Posts

    Re: Help - is a collection of studies normally distributed if the n >150? false posit

    I a meta-analysis, one doesn't count significant
    results and then calculates how many of them
    are false. Admittedly, I do not know what such
    an analysis could be good for.

    In a meta-analysis, one tries to synthesize the
    statistical information (effects sizes measures)
    from single studies, which are based on samples,
    in order to estimate the effect in the underlying
    population. So I am not sure if you really want
    to perform a meta-analysis, or perhaps something
    different.

    With kind regards

    K.

  5. #5
    Points: 3,506, Level: 37
    Level completed: 4%, Points required for next Level: 144

    Posts
    129
    Thanks
    3
    Thanked 26 Times in 26 Posts

    Re: Help - is a collection of studies normally distributed if the n >150? false posit


    If Sustat has some number of trials, regardless of the nature of the analyses or the sample size, if each test is done at alpha level 0.05, we would indeed expect 5% significant by chance if all null hypotheses were true. I think you could compare the observed percentage to that. Since these are all different topics, there is no basis for combining them. I assume you want some very general idea what percentage of studies of this sort have findings beyond chance. Say 20% of them are significant at 0.05 or less. I think it is legit to subtract 5% and say about 15% have non-type I error significance. This kind of thing is often done with genetics studies with thousands of analyses. Look up "False Discovery Rate" to learn more.

    Just remember that small studies that are not significant often don't get published (file drawer bias). But you say these are all large N. Good.

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats