Any takers? It seems like it's probably a fairly easy thing to do, I'm just a bit inexperienced at using stats (more of a qualitative guy up until this point!)
Hi, I'm having a little bit of trouble with some research I'm doing and wondered if anyone could help.
I'm looking at political decision-making. In my dataset I have 15 different countries and for 300 negotiations between these countries, I have a value between 0 and 100 indicating what "position" a country took on each negotiation. So for example the most left-wing position a country could take would get a value of 0, and the most right-wing position a country could take would get a value of 100.
What I am measuring is whether small states are more likely to be closer to the mean position in a given negotiation than large states. The theory being that larger states have more negotiating power and can afford to take more "extreme" (i.e. further away from the mean) positions than smaller states, which have less power.
This is relatively easy to work out. First I calculated the distance from the mean position for every state in every negotiation. I then separated them into two groups (10 small states and 5 large states) and did 50 paired t-tests between the small and large states using these "distance from the mean" values to produce a table that looks like this:
------------- Large State 1 ---- Large State 2 ---- Large State 3
Small State 1 ----5.6----------------6.8----------------7.2------
Small State 2 ----7.3----------------2.5----------------6.9------
...and so on (except there are 10 small states and 5 large states). In the table the value is the small state's average "distance" from the mean minus the large state's average distance. So a positive value indicates that on average the small state was closer to the mean value in the negotiation than the large state (i.e. it had a less extreme position than the large state it is compared with).
I've done this and done the usual significance adjustments (bonferroni correction, etc.) and it pretty much matches my hypothesis entirely. Almost all of the comparisons show the small state closer to the mean than the large state, and more than 20 of these comparisons are statistically significant even with the somewhat conservative bonferroni correction.
What I want to do, however, is to put a significance value on this. Rather than just pointing at the table and saying "look, almost all of the values are positive, therefore small states are more likely to be closer to the mean" I want to do a test for significance to prove that. That's where the problem comes in - I really don't know what test is the best to use.
What I thought about doing is simply creating two new columns of data. These would be the average distance from the mean for all of the 10 small states combined on every negotiation, and the average distance for all of the 5 large states combined. You could then do another paired t-test using this data and get a significance value. However, the problem with this is that it seems to be skewed because there are more small states than large states. In other words, you would expect that a group of 10 states (out of a group of 15) would be closer to the mean than the group of 5 states - because these 10 states "make up" more of the mean value.
Is there anything I could do that doesn't fall foul of this problem?
Any takers? It seems like it's probably a fairly easy thing to do, I'm just a bit inexperienced at using stats (more of a qualitative guy up until this point!)
Another thing I thought of... you could create columns of data with every combination of five from the ten small states then compare every single one with the mean for the five large states. If they all came up with significance values of 0.0000 then I'd have proved the point... seems a lot of work though given it's obvious just from looking at the table that small states are closer to the mean than large states on average.
I don't understand how you did a paired t test here since the small and large states are logically in seperate groups. They are only paired if all of both groups are in both samples. Doing 50 paired t test also seems a really bad idea given the huge correction for FW error you had to do. Usually it is reccomended not to do this (that is conduct large number of t test).
Could you have as the DV the difference from the grand mean of all states and then use a dummy variable small states/large states? I would think that would show the average difference between the two groups in terms of difference from the mean and allow a signficance test. But I am not certain of that...
Also you would have only 15 cases which is not a lot for regression (or any other method). Your power will be awful.
Last edited by noetsi; 01-28-2014 at 03:58 PM.
"Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995
Tweet |