+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 16 to 19 of 19

Thread: Choosing a sample size for a pilot study

  1. #16
    TS Contributor
    Points: 12,227, Level: 72
    Level completed: 45%, Points required for next Level: 223
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,470
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Choosing a sample size for a pilot study




    Quote Originally Posted by Lowpro View Post
    Pilot studies are meant to clarify and modify the logistics of a full trial intervention, be it sample size and power, misclassification biases inherent in data acquisition, analysis methods etc. I've heard it called a feasibility

    It's not for hypothesis testing but more of a quality control stamp on methods and data acquisition before a larger study.
    But then the sample size should be irrelevant, no? At the max. it would give an idea on the standard deviation of the measured effect, but only a rough idea.

    regards
    rogojel
    Last edited by rogojel; 06-28-2013 at 02:20 AM. Reason: typo

  2. #17
    Points: 1,155, Level: 18
    Level completed: 55%, Points required for next Level: 45

    Posts
    11
    Thanks
    1
    Thanked 1 Time in 1 Post

    Re: Choosing a sample size for a pilot study

    There is a balance here. To Lowpro's excellent points, the purpose of a pilot is not to prove the hypothesis to a measured statistical certainty. Depending on the patient population a well-powered study runs northwards of 500 patients and that ain't no pilot! There is also the matter of funding in that pilots are less expensive to run and thus easier to get funded, but still not cheap.

    The other matter is that to compute the required sample size to prove a hypothesis you need to know the alpha and beta coefficients (easy) but also the size of the treatment effect and the standard deviation. With all of those data points the required sample size is simply a matter of plug and chug, and then adding in 10-15% extra to account for dropouts. However, if you have never done a human study using your intervention then how are you supposed to know the size of the effect and the range of variability? One purpose of the pilot is to calibrate these figures more precisely as well as figuring out which markers and tests are most highly correlated with the outcome. It is one thing to try 50 markers on a small trial of 30 patients, but quite another to do it on 500 patients - the cost will eat you alive. Better to identify 10 markers that are highly indicative and then collect those 10 on the larger trial. You also discover all the things you forgot about in the protocol when you try to do the analysis on the pilot data.

    So while a pilot will be many times smaller than what is needed to prove the point (unless you get extremely lucky) it has to be large enough to calibrate the unknown variables. Does 5 patients do it? Hardly. Is 200 overkill? most of the time yes. So it is a bit of an art form to pick the right number, but the sample size is never irrelevant. In my case I can't even look at the literature to get a ballpark estimate of what the effect and standard deviation might look like, hence my appeal for better ways to approach the problem.

    Lowpro is cautioning to avoid the human tendency to read too much into the tea leaves based on a limited size data set. When you see biotech stocks on Wall Street skyrocket on smallish Phase II studies, it is people that don't understand the purpose of the small trial extrapolating underpowered results and assuming that Phase III will come out the same way. That pushes the stock price higher for a while, but more often than not the results do not hold up and the company crashes when Phase III disappoints.

  3. #18
    TS Contributor
    Points: 12,227, Level: 72
    Level completed: 45%, Points required for next Level: 223
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,470
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Choosing a sample size for a pilot study

    As far as I understand, the problem is, that you will not be able to see the effect with the pilot, unless it is a very large one. So, all, the pilot can give you is a large, probably impractical, upper limit of the effect size. I am just wondering how this would help you?

    In industrial experiments, we generally do not worry about the effect in the pilot but concentrate on the feasibility as Lowpro explained. I do not think that mixing the effect measurement with the feasibility study would be a good idea, in fact, but again, I am a complete ignorant in medical studies.

    regards
    rogojel

  4. #19
    Points: 1,155, Level: 18
    Level completed: 55%, Points required for next Level: 45

    Posts
    11
    Thanks
    1
    Thanked 1 Time in 1 Post

    Re: Choosing a sample size for a pilot study


    rogojel,

    You will not be able to measure the effect size with precision in a pilot, but you will get an idea of the range and that is very useful information. Given that a large trial can cost $10 million or more, you can't (well, shouldn't) go off into large trials without some indication of the effect size which helps you calibrate the large trial size. Imagine explaining that you did a trial without any such information using a sample of 450 only to discover than it should have been 500 as you can't just go back and run another 50 patients to get to your 500. In most cases you wind up with a failed trial that has to be repeated, from scratch, with a new set of patients. Human medical studies of significant scale very rarely take less than 3 years and often 5 years or more. So while the estimate might be imprecise, it is far and away better than no estimate.

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats