+ Reply to Thread
Results 1 to 7 of 7

Thread: Which test to use to compare data sets with a small sample size?

Hybrid View

  1. #1
    Points: 17, Level: 1
    Level completed: 33%, Points required for next Level: 33

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Which test to use to compare data sets with a small sample size?

    So I have several data sets, but each data set consists of only 7 values. I know some of the data sets are non-normal distributed (or at least I did an F-test that showed approx. half of them wasn't), so my idea was first to do a Wilcoxon Signed Rank Test. But doing it (in python) it tells me, that my sample size is quite too small. They suggest n > 20, and as stated, I only have 7 for each data set.

    So is there another test I could use instead ?

    I am comparing two treatment options. For both options some data is collected about how good and bad the treatment is. In addition, this treatment is performed once every week (7 weeks = the 7 values I mentioned) where new data is collected to see if something changes.

    So I have several data points for each patient and each treatment, which contains the effectiveness/side effects of the treatment. These different data points/values I have 7 of since they are performed one time each week.

    So basically I end up with data looking like:

    Code: 
    Patient 1, treatment 1:
    
            Side effect 1    Side effect 2    Effectiveness 1    Effectiveness 2
    Week 1     x-value          x-value           x-value            x-value
    Week 2     x-value          x-value           x-value            x-value
    Week 3     x-value          x-value           x-value            x-value
    Week 4     x-value          x-value           x-value            x-value
    Week 5     x-value          x-value           x-value            x-value
    Week 6     x-value          x-value           x-value            x-value
    Week 7     x-value          x-value           x-value            x-value
    
    
    Patient 1, treatment 2:
    
            Side effect 1    Side effect 2    Effectiveness 1    Effectiveness 2
    Week 1     x-value          x-value           x-value            x-value
    Week 2     x-value          x-value           x-value            x-value
    Week 3     x-value          x-value           x-value            x-value
    Week 4     x-value          x-value           x-value            x-value
    Week 5     x-value          x-value           x-value            x-value
    Week 6     x-value          x-value           x-value            x-value
    Week 7     x-value          x-value           x-value            x-value
    So I'm guessing I have to do some statistics on both how it changes from week to week, and then if one treatment is better than the other ?

  2. #2
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Which test to use to compare data sets with a small sample size?

    When I think of the Wilcoxon signed rank test, I think of it as the non-parametric analogy to a one-sample ttest. What are you trying to compare in the provide data example? Do you mean to do a Wilcoxon rank sum test?
    Stop cowardice, ban guns!

  3. #3
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: Which test to use to compare data sets with a small sample size?

    Quote Originally Posted by hlsmith View Post
    When I think of the Wilcoxon signed rank test, I think of it as the non-parametric analogy to a one-sample ttest. What are you trying to compare in the provide data example? Do you mean to do a Wilcoxon rank sum test?
    Just to add- Wilcoxon signed-rank test is somewhat like the paired/dependent samples t-test where as the Wilcoxon rank-sum test is like the independent samples t-test.

  4. #4
    Points: 17, Level: 1
    Level completed: 33%, Points required for next Level: 33

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Which test to use to compare data sets with a small sample size?

    Quote Originally Posted by hlsmith View Post
    When I think of the Wilcoxon signed rank test, I think of it as the non-parametric analogy to a one-sample ttest. What are you trying to compare in the provide data example? Do you mean to do a Wilcoxon rank sum test?
    I would like to see if there is any statistical "proof" of the week-to-week data is very different or not. So I guess that would be some kind of test for each data column, i.e. not comparing treatments.
    After that I would like to see if the data for treatment 1 is better than treatment 2.

  5. #5
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Which test to use to compare data sets with a small sample size?

    Well do you have a predefined distribution to compare values too. The first thing that comes to my mind is just looking at central location and dispersion, but are the values supposed to change across time?
    Stop cowardice, ban guns!

  6. #6
    Points: 17, Level: 1
    Level completed: 33%, Points required for next Level: 33

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Which test to use to compare data sets with a small sample size?

    Quote Originally Posted by hlsmith View Post
    Well do you have a predefined distribution to compare values too. The first thing that comes to my mind is just looking at central location and dispersion, but are the values supposed to change across time?
    The week to week values should optimally be the same for each week - however, this is usually not the case since many things change each week. I do have a "main value" (which is not listed here) which is the ground truth. All values SHOULD be this value if everything is perfect, but as stated, they are not, and therefore are subject to change.

  7. #7
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Which test to use to compare data sets with a small sample size?

    ondansetron,


    Yeah, I guess it was slipping my mind that is it gets used that way. I had typically used it on either difference between two groups to see if the delta value was different than 0 or a predefined value. Or use it when reviewing residuals to see if they are significantly different than "0". So in those scenarios my brain says hey, like a one-sample t-test!
    Stop cowardice, ban guns!

  8. The Following User Says Thank You to hlsmith For This Useful Post:

    ondansetron (09-07-2017)

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats