P value interpretation

[anushaka]

Hi ,

I am having a hard time understanding the concept of p value and also its interpretation. I know that I can interpret a p value of 0.80 as "the probability of a test statistic this extreme or more so less than 80 times in 100 by chance alone". But how do I interpret a p value of 1 and a p value of 0.000.

[BGM]

No matter the size of the p-value is what, you should have the same interpretation on them?

Of course in the extreme case you mentioned, you can say "if the null hypothesis is true, it is almost sure that we will not obtain a more extreme test statistic in favor of the alternative" for p-value of 0.

[noetsi]

My statistics instructors stress that it is impossible to have a p of 1 or 0 (although you can come very close) because that would mean absolute certainty and that is impossible with the p statistic. Substantively the only way I can think of you could be certain that null was or was not true is through theory. One can never determine empirically the certainty of anything, David Humes black and white swan concept explaining why. Statistics is about uncertainty not certainty.

[Dason]

My statistics instructors stress that it is impossible to have a p of 1 or 0 (although you can come very close) because that would mean absolute certainty and that is impossible with the p statistic. Substantively the only way I can think of you could be certain that null was or was not true is through theory. One can never determine empirically the certainty of anything, David Humes black and white swan concept explaining why. Statistics is about uncertainty not certainty.

I thought we already went over this - it's possible to have p = 1 or 0 - we just don't typically see if in practice because most things assume a normal distribution.

Edit: I should add that it does annoy me when I see p = 0 in homework assignments from students doing a t-test or a z-test or anova or regression. Because in those situations you can't have p = 0. It's possible have p = 1 if you're doing a two sided test in those situations so that doesn't bother me (plus that pretty much never happens so we don't need to worry too much about that).

[noetsi]

We did go over this. I disagree philisophically with your perspective. My view is that the only way you can have p = 1 and 0 is if you know something with absolute certainly with no chance that it could not be false. Which, as I argued above, is not possible by any empirical method but only through theory or by definition. Regardless of what your statistic might say, absolute certainty is not possible in any empirical method. You can not determine "truth" through data alone.

I think we are disagreeing about the possibility of absolute knowledge in emprical/statistical analysis or perhaps what p =1 or 0 logicially entails.

[Dason]

Doesn't matter. We can get p = 0 - this is mathematical fact. You can't disagree with this - it is mathematics. You can disagree with knowing things with absolute certainty but you CAN'T disagree with the fact that p can be 0.

But if it makes you feel better just remember that a probability of 0 doesn't mean that an event is impossible.

[noetsi]

I agree that given mathematical definitions you can create values they accept as 0. I disagree that they are right about the nature of reality.

As a long time academic I believe that academics defining something to be true does not make it so. And of course I am sure I could find academics like Humes that would disagree with their definitions.

Logical positivism came and went fifty years ago Dason Mabe sixty...

[anushaka]

I have dependent variable of age in yrs and two groups consisting of 98 patients in each group, with the mean age of the groups being 38.4 and 55.8, t is -8.70 and p is 0.000. My question is how do I give a verbal interpretation of the p-value for the test and is this an appropriate way of displaying a p-value?

[noetsi]

What method are you using? That is which statisical method are you using to analyze the data?

A literal interpretation of 1 for p in this case would be that there is a 100 percent probability of obtaining a value at least as extreme as the one you found if the null is true.

Note that many object to the term "if the null is true" since formally you reject or don't reject the null, but the terminology of the p uses that.

In statistical significance testing, the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.

http://en.wikipedia.org/wiki/P-value

[anushaka]

There are two questions for the first question the dependent variable is gender (m & f) and the independent variable is patient type (2 groups). there are equal no of females and equal no of males in both the groups . chi square test statatistic was used which is 0.00 and p value is 1.00., df is 1. The question is give the verbal interpret of the p value for this test statistic and is the p value consistent with the chi square calc value? explain.

[noetsi]

When you say chi square test do you mean a stand alone chi square test or a method that uses a chi square distribution such as a Wald test in logistic regression or a chi square test in SEM? I assume the former. Normally you only have a chi square statistic of 0 in very unusual cases such as having no degrees of freedom which forces chi square to be 0 in some methods (like SEM). That would be an artifact of your method rather than a true substantive result.

What was your degrees of freedom (sometimes it is listed as df or DF)?

The verbal interpretation, if it is not tied to something artificial, is what I noted above. That you are absolutely certain you should not reject the null hypotheis (whatever that is). Commonly the null is for a chi square that the two variables (gender and patient type) are indpendent of each other. Here you would be absolutely 100 percent certain that was true.

[CowboyBear]

We did go over this. I disagree philisophically with your perspective. My view is that the only way you can have p = 1 and 0 is if you know something with absolute certainly with no chance that it could not be false. Which, as I argued above, is not possible by any empirical method but only through theory or by definition. Regardless of what your statistic might say, absolute certainty is not possible in any empirical method. You can not determine "truth" through data alone.

I think we are disagreeing about the possibility of absolute knowledge in emprical/statistical analysis or perhaps what p =1 or 0 logicially entails.

Interesting. My take on this would be that there isn't anything wrong with saying:

-Given null hypothesis X and set of assumptions Y, the probability of observing a test statistic this extreme is zero (where the assumptions might apply to things like the distribution of the test statistic, the nature of sampling or randomisation employed, our faith in the honesty of the data collector, yada yada yada)
-But this doesn't mean that we have absolute confidence that null hypothesis X is untrue in the real world (because our assumptions may be incorrect).

Sorta reminds me of how Popper talked about auxiliary hypotheses and how their necessity tends to lead to some ambiguity even when a research hypothesis seems to have been falsified.

I.e. I don't see a conflict between saying that p values can legitimately reach a value of zero, while also saying that we cannot reach absolute certainty about the truth or falsity of particular hypotheses in real life.

[CowboyBear]

Commonly the null is for a chi square that the two variables (gender and patient type) are indpendent of each other. Here you would be absolutely 100 percent certain that was true.

Hold up!! A p value of 1 does not imply that we can be certain the null is true... All we can say here is that if the null is true, then it is certain we would observe a test statistic at least this extreme (naturally, since the chi-square value is zero). It's quite possible that we could also observe a test statistic of this nature given the alternative hypothesis being true, too. If the prior probability of the null being true is small and the probability of observing this test statistic under the alternative is large, the posterior probability of the null being true might still be very small (i.e. even after observing this p value).

[noetsi]

How could it be the case (since the null and alternative hypothesis by definition can not be both true) that you could see a statistic that supports both (and not just supports both but it is certain (for both of them) if they are true you would be certain to find a statistic at least this extreme)? Logically as a statistic moves in a given direction one of the two types of hypothesis is more likely, it it hard to imagine a space that a real value could hold that would fullfil this condition.

I have to admit I don't work with priors or posterior probabilities (I don't work with baysian analysis) so that part of the comments went by me

[Dason]

I'm not sure what you're trying to say noetsi - it didn't seem too clear to me. However, A p-value of 1 doesn't mean that we're certain a hypothesis is true - it just means that the data collected doesn't provide any evidence against the null hypothesis. If I get a sample mean of 50 and my null hypothesis is that the mean is 50 - I will get a p-value of 1. But if the true mean was 50.0000001 then a sample mean of 50 is pretty darn reasonable (for anything but a very large sample size and/or small standard deviation).