test whether 2 mean scores are significantly different

#1
I have two groups of participants; 1 group received treatment A and the other group received treatment B. I have the mean scores of the effectiveness for both groups.

Group A mean score before treatment was 35 and after treament 20
Group B mean score before treatment was 25 and after treatment it was 21

The lower the mean score after treatment means the patient becomes better (more effective treatment).

Clearly, treatment A works better.. but how do I test whether the mean scores are significantly different on SPSS (PASW)? Do I use paired T-test?

Thank you!
 

Link

Ninja say what!?!
#4
The fact that group B had a lower score PRE-treatment is not a good sign. Also, the fact that the scores are lower post-tx does not mean that the treatment actually helped. The conditions have to be exactly the same (i.e. the person could have had a lower measure the second time regardless of whether they were treated or not).
 

Dason

Ambassador to the humans
#5
The fact that group B had a lower score PRE-treatment is not a good sign.
That's why I asked about the randomization. But we also don't know anything about the standard deviations and sample sizes. The difference between those might be right in line with "no difference" if the sample sizes are small and there is a relatively large standard deviation for each group.

... in which case we probably wouldn't find evidence of any effect.
 
#6
thank you for your replies! :)

i understand what you are saying but how can i test if there's any significant difference between the two treatments???
 
#7
If you have everyone's individual scores, I believe you can calculate the standard deviations and use the sample size in each group to run an independent samples t-test between groups using each individual's difference score (pre minus post) as a single value and comparing the mean of differences in each group.

The starting difference between groups (10) seems pretty big compared to the observed change in means for either group (15,4), suggesting that sample size might have been on the low side (otherwise the groups should have had a more similar pre-test score, since they were randomized from the same population). Or the measurement instrument is not very fine-grained/valid/reliable, though large sample size and randomization should make that effect a wash.

Even with small sample size, you might be able to detect a significant difference (if one exists) by making use of the SD and sample size for statistical calculations. Make sure you meet the assumptions of the test you'll be using (or at the least the assumptions that it is not robust to deviations from).
 

Link

Ninja say what!?!
#8
You're welcome.

You can't test for a significant difference if the groups are different to begin with. It'd be like comparing apples to oranges.
 
#9
Well, no two groups will ever start *exactly* the same, even with large sample sizes and randomization. The question is whether they start *too* different to be able to compare directly.

But also, you can start with two groups that are significantly different and still potentially get some useful info by comparing difference scores. It doesn't give you the info you really want, but it can potentially give you some useful info. If Group A starts at 40 and Group B at 100, then Group A (Treatment X) goes down 3 and Group B (Treatment Y) goes down 65, we can compare the mean difference scores and get significance to go along with that huge effect size. We *can't* say Treatment X is less effective, but we can say that the interaction of that treatment with that type of group is less effective than the interaction between Group B and its treatment. Then it requires some non-numerical, contextual interpretation of the extent to which we can generalize. It's not perfect, but in some circumstances, it's all you can do. Then it comes down to replication to see how reliable and generalizable the results are.

*shrug*
 

Link

Ninja say what!?!
#10
Well, no two groups will ever start *exactly* the same, even with large sample sizes and randomization. The question is whether they start *too* different to be able to compare directly.

But also, you can start with two groups that are significantly different and still potentially get some useful info by comparing difference scores. It doesn't give you the info you really want, but it can potentially give you some useful info. If Group A starts at 40 and Group B at 100, then Group A (Treatment X) goes down 3 and Group B (Treatment Y) goes down 65, we can compare the mean difference scores and get significance to go along with that huge effect size. We *can't* say Treatment X is less effective, but we can say that the interaction of that treatment with that type of group is less effective than the interaction between Group B and its treatment. Then it requires some non-numerical, contextual interpretation of the extent to which we can generalize. It's not perfect, but in some circumstances, it's all you can do. Then it comes down to replication to see how reliable and generalizable the results are.

*shrug*
My view: It still won't be believable to your professional peers.