+ Reply to Thread
Results 1 to 6 of 6

Thread: Equivalency testing - CI approach and TOST don't match

  1. #1

    Equivalency testing - CI approach and TOST don't match




    Hey everyone,

    I'm doing some equivalency testing with both the CI approach and TOST to demostrate the equivalency of two scanning methods. For most of my samples, results align as one would expect, i.e. if the CI lies within the upper and lower bounds, the p-values calculated with two one-sided t-test are <0,05. However, I have two samples, where it doesnt't match at all. CIs are outside the upper and lower bounds, but p < 0,001. I wondered whether that might be because the distribution of the values is not normal and skewed to the left (mean difference is -0,27, lower bound -0,09, upper bound 0,09, 90% CI -0,418;-0,128).

    Does anyone have an idea what I'm doing wrong?

    Thanks!

  2. #2
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Equivalency testing - CI approach and TOST don't match

    hi,
    could you give a bit more details? How do you build your samples, what do you test exactly, what is the research question...

    regards

  3. The Following User Says Thank You to rogojel For This Useful Post:

    Charlotte123456 (06-12-2016)

  4. #3
    Omega Contributor
    Points: 38,413, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    7,004
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Equivalency testing - CI approach and TOST don't match

    these arent too hard to calculate by hand. give that a try and report how big of an effect size is of interest.
    Stop cowardice, ban guns!

  5. The Following User Says Thank You to hlsmith For This Useful Post:

    Charlotte123456 (06-12-2016)

  6. #4
    Points: 18, Level: 1
    Level completed: 35%, Points required for next Level: 32

    Posts
    3
    Thanks
    3
    Thanked 0 Times in 0 Posts

    Re: Equivalency testing - CI approach and TOST don't match

    Thanks for your replies!

    I have measured the same thing (muscle mass) with two different methods. The research hypothesis is that they are equivalent. N=77. I calculated the 90% Confidence Interval for the differences (-0.4180%; -0.1275%), as well as the mean difference of both methods (-0,2729%) and its standard deviation (0,77%).

    I calculated the upper and lower bounds as "mean of both methods (i.e. the mean of the 154 values) * (-)0,05", which gives me a lower bound of -0,0959% and an upper bound of 0,0959%, thus the two methods are not equivalent. That result is finde with me.

    When I went on to calculate a p-value for this, using the two one-sided tests approach. I came up with a t statistic of -2,030 for the lower bound (-0,2729--0,0959)/(0,77/SQRT(77)) and -4,229 (-0,2729-0,0959)/(0,77/SQRT(77)) for the upper bound, giving me p-values of 0,0459/2=0,02295.

    Thanks for your help!

  7. #5
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Equivalency testing - CI approach and TOST don't match

    so,
    you have a sample 1 - and a measurement with method A and a measurement with method B, then sample 2 with an Aband a B measurement and so on untill sample 77? I think the best would be to simply use a paired t-test in this case and the Bland-Altman plot.

    What are the bounds that you are calculating and what question do they answer?

  8. The Following User Says Thank You to rogojel For This Useful Post:

    Charlotte123456 (06-12-2016)

  9. #6
    Points: 18, Level: 1
    Level completed: 35%, Points required for next Level: 32

    Posts
    3
    Thanks
    3
    Thanked 0 Times in 0 Posts

    Re: Equivalency testing - CI approach and TOST don't match


    Thank you for your help!

    I think I cannot use a paired t-tet becaus I want to show that there is NO significant difference and the paired t-test would only show that there is no evidence to rejecet that hypothesis, not that it is true.

    I figured it out now - I forgot to adjust for the signs of the t-statistics ans thus got the wrong p-values. Thanks again for your help!

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats