Reply With Quote
you did the 2 methods on the same 30 people right?, then you would possibly want to test whether the two means differ on not. Thsi si called paired samples t-test. Do you have spss or excel at least 2003?
Hi guys
I'm trying to compare the differences in the results of a fitness test done by two different methods (and performed on a sample group of about 30 people). These two methods are supposed to produce very similar results, and I'm trying to find a statistical way to prove that there is non-significant difference between the results from each method. Basically the two methods are used to find the same value (maximum oxygen uptake) in each of the participants. I'm not sure what I'm meant to due mathematically with each set of data in order to determine if the difference between them is significant. I was hoping you guys could help me out!
Thanks in advance
you did the 2 methods on the same 30 people right?, then you would possibly want to test whether the two means differ on not. Thsi si called paired samples t-test. Do you have spss or excel at least 2003?
Yeah they were on the same participants. If i determine the differences between the means is there a certain value or percentage difference that would make the difference "significant" or "not-significant". See in the study below:
This is basically what I'm trying to figure out, the study mentions that the difference is "statistically significant" for 2 different methods. I'm assuming that P may refer to probability? How do I calculate this and thus determine the significance?there was some decrease in the average pulse rate
from Day 1 to Day 5 (Table 2), this difference being
statistically significant for the treadmill and the
step test (P < 0.001) but not for the bicycle ergometer.
You need to be careful here. Tests of hypothesis (like the independent samples t-test, which is probably what you want) can only be used to show that there is a significant difference between the means of the two groups. They can't, however, show that there is a nonsignificant difference between them.Originally Posted by Sinkers
If you really want to make the claim that there is no difference, you should do a power analysis. Specifically, you should do a sample size calculation based upon a desired power of the test. However, since you already took the samples, it's not going to do much good to know how many samples you should have taken (unless the number ends up being less than 30 or you can go take more samples).
This tool will give you a sample size calculation based on a desired power:
http://www.stat.ubc.ca/~rollin/stats/ssize/n2.html
oh, they were on the same 30 participants? Then you want a paired-samples t-test.
The "p" is referring to a p-value.
http://en.wikipedia.org/wiki/P-value
squareandre, i do not think the person is going to do a powr analysis with all due respect. and also, in a conference (recent) a guy showed sthing really good. the p-value is inversely related to the sample size, the larger the smaple size the more cnhances the p-value will be less than 0.05. so it is also a matter of the effect size, for which he showed a better way improved. so power analysis does good, yes up to a point, but even so, this whole thing is probably beyond this person's knowledge.
Thanks for your help guys, so I guess a paired-samples t-test is the way to go?
Yeah, I realize that now, but he made his post asking what the "p" meant while I was typing my response.
That is assuming the alternative hypothesis is true. With probability one, any difference between the means will cause the null hypothesis to be rejected for a large enough sample size. If the means actually are identical, the p-value will be independent of the sample size.
Maybe you should read some articles about criticism of hypothesis testing and statistical significance v/s practical significance.
As squareandrare said: "any difference between the means will cause the null hypothesis to be rejected for a large enough sample size".
With a enough sample size you can ALWAYS prove anything, you can always find a "difference" between method A and Method B, the important question is:
Does this observed difference have practical importance?
Many statistician are using confidence interval insted of hypothesis testing.
wipeout i agree with what you say totally. i think that is why the size effect exists, to see how big is the difference relatively to the standard deviation. but now there are better measures for this. and there is also a paper, not old enough with the title: a larger sample size is not always better in not a big journal, probably not even a purely statistical journal.
Here are the results:
In females it was found that the mean difference (M = 8.33, SD = 7.65) between the results of each test was not significantly greater than zero (t(37) = 0.98, two-tail P = 0.33, 95% CI = 5.78, 10.88). In males it was found that the mean difference (M = 9.18, SD = 7.48) between the results of each test was also not significantly greater than zero (t(38) = -0.68, two-tail P = 0.50, 95% CI = 6.75, 11.6).
I used a guide and performed this in excel and to be honest have very little idea of what any of those values mean (besides Mean, SD and CI). Am I right in thinking that the results are not-significantly greater than zero?
yes they are non-signficant, the P=0.33 and P=0.5 for the females and males respectively are the p-values. Since they are over 0.05 (default value) you report that there are not evidence to support that the means for the males and the females separately differ significantly at a level of significance equal to 0.05.
I'm sorry but I don't agree with last conclusion.
Are the two fitness methods different?
Of course, they are. It's impossible that two things are equals. With more sample size you can always reject H0, it doesn't matter if the difference between both groups is small.
What do you mean with no differences? Do you mean practical differences? or Do you mean statistical differences?
What do you think about this: Are they equals?
Mean Method A v/s Mean Method B
8.300000000000000 8.300000000000001
I bet if you have a very big sample size you can prove that there is difference.
Please, visit this pages:
The Insignificance of Statistical Significance Testing
http://www.npwrc.usgs.gov/resource/m...ig/stathyp.htm
A serious statistician would use confidence interval, effect size, bayesian approach or bootstrap instead of a single statistical hypothesis
wipeout, the guy is not a statistician, that is why he came here. No need to make him live before his time. he has only 30 males and 30 females. he did two tests, that's it. as i said before, there is no evidence to support the means differ. if he had 100000 people ok, but he has not. let's keep it simple and try to help.
Posting Permissions
|
|
