Reverse Psychology

[psychgradstudent]

I have a rather unique problem. All the hypotheses I've come across in psychology statistics test for a significant difference between two or more groups. However, I need to test for the absence of a significant difference between two groups. In other words, my hypothesis states that there is no significant difference between my two groups.

I am using the Wilcoxon Signed-Rank Test. I know how to run this particular test, but I am unclear on one point.

The typical hypothesis states that a difference exists between the groups. In this case, the researcher rejects the null hypothesis when the test value is equal to or less than the critical value.

Does that mean that in my case, where my hypothesis states that a difference does not exist, I reject the null hypothesis if the test value is greater than the critical value?

[staryeyed]

Remember that when you are thinking in terms of the null hypothesis, the null hypothesis is that there is no effects (A=B). The null is the same regardless of what you think will happen. So if you think that the null hypothesis is true then a p value which is larger than the critical value would still lead you to fail to reject the null, the only difference is that a statistic which causes you to fail to reject the null is in support of your hypothesis.
Attempting to show support for a null hypothesis is difficult because, as you said, your statistical tests just aren't built to show that there is no difference. The way I usually see this problem handled (in psychology papers) is by acknowledging that it is not possible to prove a null result but to then emphasize why in your case it seems to be the best explanation of the current findings. To do this you want to show that the reason for the null finding is:
1.) based on good theory- you can explain both your findings and the findings of others in a way that makes sense in the context of the current literature,
2.)based on good statistics- your study has very high power so that the null result is not just for lack of the correct number of participants, and
3.) that your test came up far from significant- just use the standard test and show that p is above .05 (hopefully quite a bit above).

[psychgradstudent]

Thank you so much for your reply. It was very enlightening and gave me direction for further reading on my particular problem.

Maybe I'm going about this the wrong way. Perhaps someone has a better idea of how to tackle my research statistically.

For my masters thesis, I developed a geographic profiling program which mathematically models the psychology serial offenders use when selecting crime locations. The user inputs a series of crime scene locations and the program outputs a density map indicating areas with a high probability of containing the serial offender's home residence.

Three established programs of this nature existed before the development of my program. I am attempting to establish the reliability of my program (in terms of accuracy) for my thesis. In other words, I want to show that my program works as well as the established programs.

Accuracy is measured in terms of "search cost" or the percentage of the map searched before finding the actual home location of the serial offender. I ran the same set of 55 crime series through each program and recorded the search cost. Due to sampling technique, I must use a nonparametric test.

My initial instinct was to compare my program with each of the established programs using a Wilcoxon Signed-Rank test to test for no difference. Obviously, as discussed above, this presents some challenges.

Does anyone have any other ideas of how I can statistically establish the reliability of my program (to show that mine works as well as theirs)?

[gearl42]

I can't point to a specific procedure, but it seems to me that your problem is similar to the one that the developers of a new drug face when they must establish that their drug is no less effective than known treatments. Since this is a fairly routine task, you should be able to find relevant statistical measures in the medical literature.

[JohnM]

It's referred to as "Equivalence Testing" but is used with parametric statistical procedures. At the risk of oversimplifying, basically a "delta" or max tolerable difference is determined and then the confidence interval for the difference needs to fall entirely within this delta in order to conclude equivalence. There may be a way to demonstrate equivalence of two "things" with ranks, but I'm not aware of any....

[psychgradstudent]

Well, I am assuming I have to use an nonparametric test. My data is interval in nature, but the series used to get that data were not randomly selected. Allow me to explain.

I started with a data set from another researcher consisting of 88 series of crimes. Each series consists of the lat/lon coordinates for a number of crime scenes; the number of scenes varies from series to series. Each series also contains the lat/lon coordinates of the criminal's home residence. I am not exactly sure how these cases were chosen for the set, but they were most likely selected based on their status as solved cases committed by a single criminal living at one location for the duration of the series.

I then selected 55 series from the set of 88 based on criteria recommended by researchers for conducting my type of research. This basically consisted of eliminating series where the home residence is significantly outside the convex hull of the crime scenes.

The actual data I am testing consists of an accuracy measure called the "search cost". Basically, it is the percentage of the map each program has to search before finding the actual home residence.

Am I limited to nonparametric tests because of the way I selected certain series to be included in the research? Or can I employ parametric tests because I did not select the series based on the the actual data I am analyzing (search costs) and had no idea what those would be while building the data set?

I apologize for the continuing and lengthy treatise on my problem, but I greatly appreciate all of your help!

UPDATE: I did a normality test for my data and found it is far from normal. So I guess I'm limited to nonparametric tests after all. However, if anyone has parametric data and needs to test for equivalence, you may find this link very helpful: http://www.graphpad.com/library/Bios...rticle_182.htm

[psychgradstudent]

I apologize for posting to my thread once again. This will be the last time!

After some hours of research, I found that statisticians sometimes transform their data using the log function in order to obtain a normal distribution. So...

1. Is that a statistically valid procedure?

2. Assuming I can meet the normality assumption, can I run a parametric test on my data? (See the above post for a description of my data.)

[JohnM]

Yes, transforming is a valid method, but you eventually run into the complication of practical interpretation of results......which is why I don't like to transform....