+ Reply to Thread
Results 1 to 4 of 4

Thread: Question about sample size penalty for looking at data mid-trial

  1. #1
    Points: 7, Level: 1
    Level completed: 13%, Points required for next Level: 43

    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Question about sample size penalty for looking at data mid-trial




    Hi, we're running a clinical trial for a skin treatment with a medical device that has a strong effect. We are using an inter-patient control (no treatment arm) vs active treatment arm and estimate we need 8 patients enrolled for significance. That being said, for marketing reasons, we're going to do 30 patients.

    I'm the CEO and would like to take a look at the data after 15 patients to help with fundraising. I would keep the investigators blinded to the results and would not use the data for any decisions (stopping or continuing). For this reason, I don't think its an interim analysis. Do I need to decrease my p-value at the 15 or 30 analysis? I'm trying to argue w/ my statistician that I don't (I'm losing right now).

    If so, what is a reasonable approach to use? Obrien-Flemming seems to be too conservative to me at the first look. Should I try a different approach? If so which one?

  2. #2
    Points: 6,387, Level: 52
    Level completed: 19%, Points required for next Level: 163
    Junes's Avatar
    Location
    Netherlands
    Posts
    246
    Thanks
    17
    Thanked 25 Times in 20 Posts

    Re: Question about sample size penalty for looking at data mid-trial

    Hi, welcome.

    That's an interesting problem. In principle, I think there is nothing wrong with looking at the results ad interim, even with null hypothesis testing. As long as you stick to your plan and don't base your decisions on it. And on the one hand, you don't. There's nothing necessarily wrong with looking or even communicating preliminary results, I think.

    However, when you're talking about p-values and significance, you're inevitably in testing territory, and other rules apply. Most likely, you might not claim non-significance at the fundraiser with a result of, say, p = .3 (perhaps say something about "interesting preliminary results", giving your audience a chance to hear about the "significance" after the study), whereas you might be tempted to proclaim a result of p = .03 "significant". Thus, for a member of the audience at the fundraiser, their "internal Type-I error" (so to speak) is inflated.

    Personally, I would just report preliminary effect sizes but leave out all talk of p-values.

  3. #3
    TS Contributor
    Points: 12,227, Level: 72
    Level completed: 45%, Points required for next Level: 223
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,470
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Question about sample size penalty for looking at data mid-trial

    hi,
    I would take the side of your statistician here.The point of the sample size calculation is to make sure you have enough data to draw reliable conclusions from. This also means that smaller sample sizes are unreliable, essentially random numbers. How would random numbers help you?

    And this is only the most obvious problem - even with the best of intentions a CEO looking at intermediate results would be a huge warning sign to me - there are so many unconscious signals that influence an experiment....

    regards

  4. #4
    Omega Contributor
    Points: 38,326, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,995
    Thanks
    398
    Thanked 1,185 Times in 1,146 Posts

    Re: Question about sample size penalty for looking at data mid-trial


    Agreed. Early review of data is typically underpowered or runs the risk of random sampling not doing what it is suppose to. Meaning by chance sampling variation may have placed the first extreme case into one of the two groups and the 50% probability of treatment balance for covariates may not have occurred yet. Subconsciously if the results are great you embrace them and if not you just say it is too early, which isn't rational.
    Stop cowardice, ban guns!

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats