Hi all,
I'm a grad student who uses stats a lot but who has very little formal training. Not sure if this is in the right forum, so feel free to move it if that sort of thing is done here.
Here's a question that comes up a lot and I've never had answered very well. Basically, I often have two distributions and want to assess whether they're similar. Say they're normally distributed and a suitable test for dissimilarity would be a t-test.
Now, is it acceptable to simply perform a t-test, and then use a p-value>.05 as a criteria for similarity? This is usually my first reaction. But in practice this seems way too liberal, since p>.05 corresponds to 95% of the distribution - whereas p<.05 corresponds to 5%. Thus testing for p>.05 tends to admit way too much 'noise' and my numbers are usually hopelessly skewed upwards.
If what I want is to select the distribution pairs (normally distributed) that are similar, what's an appropriate test?
I'm a grad student who uses stats a lot but who has very little formal training. Not sure if this is in the right forum, so feel free to move it if that sort of thing is done here.
Here's a question that comes up a lot and I've never had answered very well. Basically, I often have two distributions and want to assess whether they're similar. Say they're normally distributed and a suitable test for dissimilarity would be a t-test.
Now, is it acceptable to simply perform a t-test, and then use a p-value>.05 as a criteria for similarity? This is usually my first reaction. But in practice this seems way too liberal, since p>.05 corresponds to 95% of the distribution - whereas p<.05 corresponds to 5%. Thus testing for p>.05 tends to admit way too much 'noise' and my numbers are usually hopelessly skewed upwards.
If what I want is to select the distribution pairs (normally distributed) that are similar, what's an appropriate test?