Equivalency testing - CI approach and TOST don't match

[Charlotte123456]

Hey everyone,

I'm doing some equivalency testing with both the CI approach and TOST to demostrate the equivalency of two scanning methods. For most of my samples, results align as one would expect, i.e. if the CI lies within the upper and lower bounds, the p-values calculated with two one-sided t-test are <0,05. However, I have two samples, where it doesnt't match at all. CIs are outside the upper and lower bounds, but p < 0,001. I wondered whether that might be because the distribution of the values is not normal and skewed to the left (mean difference is -0,27, lower bound -0,09, upper bound 0,09, 90% CI -0,418;-0,128).

Does anyone have an idea what I'm doing wrong?

Thanks!

 

[rogojel]

hi,
could you give a bit more details? How do you build your samples, what do you test exactly, what is the research question...

regards

 

[hlsmith]

these arent too hard to calculate by hand. give that a try and report how big of an effect size is of interest.

 

[Charlotte123456]

Thanks for your replies!

I have measured the same thing (muscle mass) with two different methods. The research hypothesis is that they are equivalent. N=77. I calculated the 90% Confidence Interval for the differences (-0.4180%; -0.1275%), as well as the mean difference of both methods (-0,2729%) and its standard deviation (0,77%).

I calculated the upper and lower bounds as "mean of both methods (i.e. the mean of the 154 values) * (-)0,05", which gives me a lower bound of -0,0959% and an upper bound of 0,0959%, thus the two methods are not equivalent. That result is finde with me.

When I went on to calculate a p-value for this, using the two one-sided tests approach. I came up with a t statistic of -2,030 for the lower bound (-0,2729--0,0959)/(0,77/SQRT(77)) and -4,229 (-0,2729-0,0959)/(0,77/SQRT(77)) for the upper bound, giving me p-values of 0,0459/2=0,02295.

Thanks for your help!

 

[rogojel]

so,
you have a sample 1 - and a measurement with method A and a measurement with method B, then sample 2 with an Aband a B measurement and so on untill sample 77? I think the best would be to simply use a paired t-test in this case and the Bland-Altman plot.

What are the bounds that you are calculating and what question do they answer?

 

[Charlotte123456]

Thank you for your help!

I think I cannot use a paired t-tet becaus I want to show that there is NO significant difference and the paired t-test would only show that there is no evidence to rejecet that hypothesis, not that it is true.

I figured it out now - I forgot to adjust for the signs of the t-statistics ans thus got the wrong p-values. Thanks again for your help!