+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 23

Thread: P value interpretation

  1. #1
    Points: 558, Level: 11
    Level completed: 16%, Points required for next Level: 42

    Posts
    3
    Thanks
    3
    Thanked 0 Times in 0 Posts

    Question P value interpretation




    Hi ,

    I am having a hard time understanding the concept of p value and also its interpretation. I know that I can interpret a p value of 0.80 as "the probability of a test statistic this extreme or more so less than 80 times in 100 by chance alone". But how do I interpret a p value of 1 and a p value of 0.000.

  2. #2
    TS Contributor
    Points: 22,410, Level: 93
    Level completed: 6%, Points required for next Level: 940

    Posts
    3,020
    Thanks
    12
    Thanked 565 Times in 537 Posts

    Re: P value interpretation

    No matter the size of the p-value is what, you should have the same interpretation on them?

    Of course in the extreme case you mentioned, you can say "if the null hypothesis is true, it is almost sure that we will not obtain a more extreme test statistic in favor of the alternative" for p-value of 0.

  3. The Following User Says Thank You to BGM For This Useful Post:

    anushaka (11-23-2011)

  4. #3
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: P value interpretation

    My statistics instructors stress that it is impossible to have a p of 1 or 0 (although you can come very close) because that would mean absolute certainty and that is impossible with the p statistic. Substantively the only way I can think of you could be certain that null was or was not true is through theory. One can never determine empirically the certainty of anything, David Humes black and white swan concept explaining why. Statistics is about uncertainty not certainty.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  5. #4
    Devorador de queso
    Points: 95,540, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,930
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: P value interpretation

    Quote Originally Posted by noetsi View Post
    My statistics instructors stress that it is impossible to have a p of 1 or 0 (although you can come very close) because that would mean absolute certainty and that is impossible with the p statistic. Substantively the only way I can think of you could be certain that null was or was not true is through theory. One can never determine empirically the certainty of anything, David Humes black and white swan concept explaining why. Statistics is about uncertainty not certainty.
    I thought we already went over this - it's possible to have p = 1 or 0 - we just don't typically see if in practice because most things assume a normal distribution.

    Edit: I should add that it does annoy me when I see p = 0 in homework assignments from students doing a t-test or a z-test or anova or regression. Because in those situations you can't have p = 0. It's possible have p = 1 if you're doing a two sided test in those situations so that doesn't bother me (plus that pretty much never happens so we don't need to worry too much about that).

  6. #5
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: P value interpretation

    We did go over this. I disagree philisophically with your perspective. My view is that the only way you can have p = 1 and 0 is if you know something with absolute certainly with no chance that it could not be false. Which, as I argued above, is not possible by any empirical method but only through theory or by definition. Regardless of what your statistic might say, absolute certainty is not possible in any empirical method. You can not determine "truth" through data alone.

    I think we are disagreeing about the possibility of absolute knowledge in emprical/statistical analysis or perhaps what p =1 or 0 logicially entails.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  7. #6
    Devorador de queso
    Points: 95,540, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,930
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: P value interpretation

    Doesn't matter. We can get p = 0 - this is mathematical fact. You can't disagree with this - it is mathematics. You can disagree with knowing things with absolute certainty but you CAN'T disagree with the fact that p can be 0.

    But if it makes you feel better just remember that a probability of 0 doesn't mean that an event is impossible.

  8. #7
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: P value interpretation

    I agree that given mathematical definitions you can create values they accept as 0. I disagree that they are right about the nature of reality.

    As a long time academic I believe that academics defining something to be true does not make it so. And of course I am sure I could find academics like Humes that would disagree with their definitions.

    Logical positivism came and went fifty years ago Dason Mabe sixty...
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  9. The Following User Says Thank You to noetsi For This Useful Post:

    anushaka (11-23-2011)

  10. #8
    Points: 558, Level: 11
    Level completed: 16%, Points required for next Level: 42

    Posts
    3
    Thanks
    3
    Thanked 0 Times in 0 Posts

    Re: P value interpretation

    I have dependent variable of age in yrs and two groups consisting of 98 patients in each group, with the mean age of the groups being 38.4 and 55.8, t is -8.70 and p is 0.000. My question is how do I give a verbal interpretation of the p-value for the test and is this an appropriate way of displaying a p-value?

  11. #9
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: P value interpretation

    What method are you using? That is which statisical method are you using to analyze the data?

    A literal interpretation of 1 for p in this case would be that there is a 100 percent probability of obtaining a value at least as extreme as the one you found if the null is true.

    Note that many object to the term "if the null is true" since formally you reject or don't reject the null, but the terminology of the p uses that.

    In statistical significance testing, the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.
    http://en.wikipedia.org/wiki/P-value
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  12. The Following User Says Thank You to noetsi For This Useful Post:

    anushaka (11-23-2011)

  13. #10
    Points: 558, Level: 11
    Level completed: 16%, Points required for next Level: 42

    Posts
    3
    Thanks
    3
    Thanked 0 Times in 0 Posts

    Re: P value interpretation

    There are two questions for the first question the dependent variable is gender (m & f) and the independent variable is patient type (2 groups). there are equal no of females and equal no of males in both the groups . chi square test statatistic was used which is 0.00 and p value is 1.00., df is 1. The question is give the verbal interpret of the p value for this test statistic and is the p value consistent with the chi square calc value? explain.

  14. #11
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: P value interpretation

    When you say chi square test do you mean a stand alone chi square test or a method that uses a chi square distribution such as a Wald test in logistic regression or a chi square test in SEM? I assume the former. Normally you only have a chi square statistic of 0 in very unusual cases such as having no degrees of freedom which forces chi square to be 0 in some methods (like SEM). That would be an artifact of your method rather than a true substantive result.

    What was your degrees of freedom (sometimes it is listed as df or DF)?

    The verbal interpretation, if it is not tied to something artificial, is what I noted above. That you are absolutely certain you should not reject the null hypotheis (whatever that is). Commonly the null is for a chi square that the two variables (gender and patient type) are indpendent of each other. Here you would be absolutely 100 percent certain that was true.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  15. #12
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: P value interpretation

    Quote Originally Posted by noetsi View Post
    We did go over this. I disagree philisophically with your perspective. My view is that the only way you can have p = 1 and 0 is if you know something with absolute certainly with no chance that it could not be false. Which, as I argued above, is not possible by any empirical method but only through theory or by definition. Regardless of what your statistic might say, absolute certainty is not possible in any empirical method. You can not determine "truth" through data alone.

    I think we are disagreeing about the possibility of absolute knowledge in emprical/statistical analysis or perhaps what p =1 or 0 logicially entails.
    Interesting. My take on this would be that there isn't anything wrong with saying:

    -Given null hypothesis X and set of assumptions Y, the probability of observing a test statistic this extreme is zero (where the assumptions might apply to things like the distribution of the test statistic, the nature of sampling or randomisation employed, our faith in the honesty of the data collector, yada yada yada)
    -But this doesn't mean that we have absolute confidence that null hypothesis X is untrue in the real world (because our assumptions may be incorrect).

    Sorta reminds me of how Popper talked about auxiliary hypotheses and how their necessity tends to lead to some ambiguity even when a research hypothesis seems to have been falsified.

    I.e. I don't see a conflict between saying that p values can legitimately reach a value of zero, while also saying that we cannot reach absolute certainty about the truth or falsity of particular hypotheses in real life.

  16. #13
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: P value interpretation

    Quote Originally Posted by noetsi View Post
    Commonly the null is for a chi square that the two variables (gender and patient type) are indpendent of each other. Here you would be absolutely 100 percent certain that was true.
    Hold up!! A p value of 1 does not imply that we can be certain the null is true... All we can say here is that if the null is true, then it is certain we would observe a test statistic at least this extreme (naturally, since the chi-square value is zero). It's quite possible that we could also observe a test statistic of this nature given the alternative hypothesis being true, too. If the prior probability of the null being true is small and the probability of observing this test statistic under the alternative is large, the posterior probability of the null being true might still be very small (i.e. even after observing this p value).

  17. #14
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: P value interpretation

    How could it be the case (since the null and alternative hypothesis by definition can not be both true) that you could see a statistic that supports both (and not just supports both but it is certain (for both of them) if they are true you would be certain to find a statistic at least this extreme)? Logically as a statistic moves in a given direction one of the two types of hypothesis is more likely, it it hard to imagine a space that a real value could hold that would fullfil this condition.

    I have to admit I don't work with priors or posterior probabilities (I don't work with baysian analysis) so that part of the comments went by me
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  18. #15
    Devorador de queso
    Points: 95,540, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,930
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: P value interpretation


    I'm not sure what you're trying to say noetsi - it didn't seem too clear to me. However, A p-value of 1 doesn't mean that we're certain a hypothesis is true - it just means that the data collected doesn't provide any evidence against the null hypothesis. If I get a sample mean of 50 and my null hypothesis is that the mean is 50 - I will get a p-value of 1. But if the true mean was 50.0000001 then a sample mean of 50 is pretty darn reasonable (for anything but a very large sample size and/or small standard deviation).

+ Reply to Thread
Page 1 of 2 1 2 LastLast

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats