Evidencing the null

trinker

ggplot2orBust
#1
I remember having this same discussion a few months back and there being no resolution...

Let's say I have a repeated measures analysis with a control group (the analysis really is arbitrary). I want to show there is no difference between the control and the test group. what methods could I use for that. I'm so used to desiring for the evidence to reject the null. It seems unwise to use traditional techniques unless you set alpha higher instead of lower. Maybe a [TEX].10 = \alpha[/TEX]? Not sure how to approach this. Haven't really looked yet , but if anyone has a direction for me to investigate I'd appreciate it.

EDIT: Useful Links on this topic (added as I find them)

Why you can't "prove" the null LINK
 
Last edited:

Dason

Ambassador to the humans
#2
You can't directly test that there is no difference. What you can test is that that the difference is less than some threshold - which is known as equivalence testing (I believe). So you set some threshold that represents the smallest difference you or anybody would care about - such that if there difference was smaller than that everybody would say that for all practical purposes the values are the same. We can test that the difference is smaller than that value easily.
 

Dason

Ambassador to the humans
#4
Although I think there has been some work with Baye's factors to do what you wanted to do in the original post.
 

noetsi

No cake for spunky
#5
I wanted to clarify my confusion on this point. I thought the null of repeated measures was that there was no difference between the mean values (of whatever intervention you are testing)? Why is this not appropriate here (I am sure I will feel really **** when dason points out the reason).

Not that I doubt it is true. I just want to figure out what my misunderstanding of repeated measure ANOVA is here.
 

Dason

Ambassador to the humans
#6
The thing trinker was asking about was reversing the hypothesis. Instead of trying to prove that there is a different he wants to prove that there is no difference.
 

trinker

ggplot2orBust
#7
This has to be fairly common in clinical trials. say for instance you have a generic drug you want to show works equally as well ass the leading brand. But yes dason is precisely correct. I just happent to be doing a repeated emasures design, the problem however is with the reverse hypothsis.