+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 23

Thread: What is the difference between classical and bayesian understanding of probability?

  1. #1
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    What is the difference between classical and bayesian understanding of probability?




    I am reading a book on monte carlo analysis and the author stresses in passing that the definition of probability used in an example in the book is the classical aka frequentist one, and that Bayesian understanding of probability is different. They say the difference is too complex to explain in the book.

    The comment goes "Thus, we can define probability in terms of the relative frequency with which an event will occur if we repeated our "experiment" [a coin flip] a great many times. This is a classic "frequentist" interpretation of probability that lies at the heart of traditional or "classical" statistics." [they then to go on to point out that Bayesian's have a different interpretation of probability, but no details are provided].

    I was hoping for a brief explanation of how the two differ.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  2. #2
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    Welp there is a metric ****-ton of articles about the difference, so it'd be worth reading up, but a very simple definition would be that a frequentist interprets probability just how it's stated in your quote:

    the relative frequency with which an event will occur if we repeated our "experiment" [a coin flip] a great many times.
    This means that you can only usefully talk about the probability of events happening if you can at least imagine in principle conducting repeated "trials" (experiments, coin flips, whatever) in which sometimes the event takes place and sometimes it doesn't. To a frequentist, it's meaningless to ask what the probability it is that some hypothesis is true; the hypothesis is either true or it isn't (it isn't true in some trials and false in others). This is part of the reason why the probabilities we talk about in frequentism (e.g., p values) refer to how probable it is that a particular thing will be observed in an experiment, not how probable hypotheses themselves are. We can imagine conducting repeated experiments in which sometimes a thing happens and sometimes it doesn't. (But the hypothesis will simply remain either true or false).

    In the Bayesian interpretation of probability, a probability is a state of certainty or belief with respect to some proposition. That probability could refer to the rate at which an event (e.g., a coin flip coming up heads) occurs over repeated trials, but it doesn't have to: A Bayesian can also refer to the probability that a particular hypothesis is true.

    And Bayes theorem shows us how to combine existing knowledge/beliefs/information along with the data observed to make statements about probabilities.

    (Note: That's a super simplistic version!)
    Matt aka CB | twitter.com/matthewmatix

  3. The Following User Says Thank You to CowboyBear For This Useful Post:

    Buckeye (02-07-2017)

  4. #3
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    As a non-statistician I guess the frequentist position makes more sense to me. The sky is, to be simplistic, either blue or its not. That is hypothesis are either true or they are not since to me the hypothesis is about what is true in the population not the sample (at a given time anyhow obviously reality changes in some cases).
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  5. #4
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    Quote Originally Posted by noetsi View Post
    As a non-statistician I guess the frequentist position makes more sense to me. The sky is, to be simplistic, either blue or its not. That is hypothesis are either true or they are not since to me the hypothesis is about what is true in the population not the sample (at a given time anyhow obviously reality changes in some cases).
    Just to be clear: The Bayesian position isn't that a particular hypothesis is sometimes true and sometimes false. The Bayesian position is that we can make probability statements about things that are either true or false. So a Bayesian can say something like "there is a 95% probability that my hypothesis is true", whereas a frequentist can't.

    Anyway the best thing to do if you're interested in the difference between the two perspectives is to read more. Some sources to start with:

    Kruschke, J. K. (2010). What to believe: Bayesian methods for data analysis. Trends in Cognitive Sciences, 14(7), 293–300. https://doi.org/10.1016/j.tics.2010.05.001
    Gigerenzer, G., Krauss, S., & Vitouch, O. (2004). The null ritual: What you always wanted to know about significance testing but were afraid to ask. In D. Kaplan (Ed.), The Sage handbook of quantitative methodology for the social sciences (pp. 391–408). Thousand Oaks, CA: Sage.
    Dienes, Z. (2011). Bayesian versus orthodox statistics: Which side are you on? Perspectives on Psychological Science, 6(3), 274–290. https://doi.org/10.1177/1745691611406920
    Matt aka CB | twitter.com/matthewmatix

  6. The Following User Says Thank You to CowboyBear For This Useful Post:

    noetsi (02-07-2017)

  7. #5
    Points: 1,821, Level: 25
    Level completed: 21%, Points required for next Level: 79
    Buckeye's Avatar
    Location
    Ohio
    Posts
    102
    Thanks
    31
    Thanked 4 Times in 4 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    I have a follow up question. Because conditional probabilities and Bayes' Theorem piqued my interest as an undergraduate.

    Are these two definitions of probability always used together? In other words, is it possible for someone to think entirely in terms of Bayesian statistics and "ignore" the frequentist perspective? I don't imagine so. But I'm curious.

    I really enjoy what I've seen of Bayesian statistics.
    "I have discovered a truly remarkable proof of this theorem which this margin is too narrow to contain." Pierre de Fermat

  8. #6
    Devorador de queso
    Points: 95,819, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,935
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    One of the reasons a lot of people like the bayesian perspective is entirely because they prefer the bayesian interpretation of probability.

    Want to talk about the probability your favorite team will win the super bowl? Well try thinking about what a frequentist defines probability as and think if it makes sense to talk about the probability that the Packers (obviously this is everybody's favorite team) wins the super bowl next year. With a bayesian interpretation it's very natural to talk about probability of one time events like this. If you have this chat with a frequentist there is a non-zero probability (hah) that you'll hear something about multiple universes or hypothetical timelines and it starts to sound *really complicated* just to talk about how you can define probability for an event like that.
    I don't have emotions and sometimes that makes me very sad.

  9. #7
    Devorador de queso
    Points: 95,819, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,935
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    Quote Originally Posted by Buckeye View Post
    I have a follow up question. Because conditional probabilities and Bayes' Theorem piqued my interest as an undergraduate.

    Are these two definitions of probability always used together? In other words, is it possible for someone to think entirely in terms of Bayesian statistics and "ignore" the frequentist perspective? I don't imagine so. But I'm curious.

    I really enjoy what I've seen of Bayesian statistics.
    There are a few issues here. First note that if you're just using bayes theorem to do a probability calculation that doesn't mean you're doing bayesian statistics.

    Also it's entirely possible to be a frequentist and use bayesian methods. You might just be interested in different estimators that have nice properties and want to discuss the frequentist properties of such estimators. It's also possible to be a bayesian and and want to explore the frequentist properties of your estimators. They aren't entirely mutually exclusive schools of thought.
    I don't have emotions and sometimes that makes me very sad.

  10. The Following User Says Thank You to Dason For This Useful Post:

    Buckeye (02-07-2017)

  11. #8
    Points: 1,821, Level: 25
    Level completed: 21%, Points required for next Level: 79
    Buckeye's Avatar
    Location
    Ohio
    Posts
    102
    Thanks
    31
    Thanked 4 Times in 4 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    Okay, I have a limited knowledge under my belt. So, I naturally attributed Bayes' Theorem to that school of thought. As a side note, I'm excited to pursue a master's in the fall. Applications of statistics are endless!
    "I have discovered a truly remarkable proof of this theorem which this margin is too narrow to contain." Pierre de Fermat

  12. #9
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    While this may not be theoretically the case I commonly read analysis that mix bayseian and classical statistics. That is they use Bayesian approaches sometimes and frequentist statistics other times.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  13. #10
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    Quote Originally Posted by Dason View Post
    If you have this chat with a frequentist there is a non-zero probability (hah) that you'll hear something about multiple universes or hypothetical timelines and it starts to sound *really complicated* just to talk about how you can define probability for an event like that.
    God**** frequentists with their multiverses and superpopulations...
    Matt aka CB | twitter.com/matthewmatix

  14. #11
    TS Contributor
    Points: 22,418, Level: 93
    Level completed: 7%, Points required for next Level: 932
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    2,135
    Thanks
    166
    Thanked 537 Times in 431 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    Quote Originally Posted by Dason View Post
    If you have this chat with a frequentist there is a non-zero probability (hah) that you'll hear something about multiple universes or hypothetical timelines and it starts to sound *really complicated* just to talk about how you can define probability for an event like that.
    Are you half-kidding or is this for real? As in, does the frequentist interpretation of probability implicitly assumes the need for multiple universes/timelines? Is it something you heard or is it something that someone has tried to justify somewhere like in a journal article or book or something?
    for all your psychometric needs! https://psychometroscar.wordpress.com/about/

  15. #12
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    Quote Originally Posted by spunky View Post
    Are you half-kidding or is this for real? As in, does the frequentist interpretation of probability implicitly assumes the need for multiple universes/timelines? Is it something you heard or is it something that someone has tried to justify somewhere like in a journal article or book or something?
    Dunno about Dason, but I'm pretty sure I've seen this kind of thing. I mean, after all the winner of next year's superbowl isn't a random variable: You can't say it's 65% likely to be the Packers in the frequentist definition, because there aren't going to be multiple 2018 superbowls in which sometimes the Packers win and sometimes they don't. So to make a frequentist probability statement you have to make come up with some imaginary way in which there are multiple "trials". (Not that the frequentist interpretation implicitly requires multiverses - it just kinda requires them if we want to stretch frequentist probability into making statements about things that it's not designed to make statements about).
    Matt aka CB | twitter.com/matthewmatix

  16. The Following User Says Thank You to CowboyBear For This Useful Post:

    Dason (02-07-2017)

  17. #13
    TS Contributor
    Points: 12,227, Level: 72
    Level completed: 45%, Points required for next Level: 223
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,470
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    The simplest way to explain the randomness in quantum physics is the multiple universe interpretation of Everett - so frequentists might in the end be right after all.

    regards

  18. #14
    TS Contributor
    Points: 40,621, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Downloads
    gianmarco's Avatar
    Location
    Italy
    Posts
    1,368
    Thanks
    232
    Thanked 301 Times in 225 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit

    Quote Originally Posted by noetsi View Post
    As a non-statistician I guess the frequentist position makes more sense to me.
    I am the least qualified person who can talk about theoretical aspects of statistics in general, and of Bayesian approaches in particular. I wish to just stress that Bayesian "standpoint" has its relevance in those fields where you want to assess, for instance, how likely is your prior knowledge on the basis of what you observe, i.e. data.

    For instance, in archaeological radiocarbon dating, Bayesian statistic provides scientists with a formal way to update one's belief in the light of the data (s)he has currently in his/her hand. For example, we got radiocarbon data from context A, B, and C, and we have a prior belief/hypothesis (deriving from other independent observations) as to the fact that A is older than B, and B is older than C. Bayesian analysis allows us to arrive at the posterior probability as to the soundness of our prior hypothesis. In other words, given our prior hypothesis AND given the data, we can "update" our prior belief in the light of what data are telling us (so getting a posterior probability).

    I found a book whose first sections are good for making non-stat people (like me) grasp some basic principle of Bayesian inference:
    https://www.amazon.com/Bayes-Rule-Tu.../dp/0956372848



    gm
    http://cainarchaeology.weebly.com/

  19. The Following User Says Thank You to gianmarco For This Useful Post:

    Buckeye (02-08-2017)

  20. #15
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: What is the difference between classical and bayesian understanding of probabilit


    Right on Rogojel it is all about M-verses. String theory is so 90's for buzz terms. I am also interested in the idea of M-verses. Opens the doors to many awesome movies as well. And I keep wondering if our universe is ever expanding cant we stave off our own freezing demise with some type of human generated nuclear power. I digress, but was thinking the same thing about M-verses.


    I don't get all gummed up by the idea that given the event in a super-population is true, that realization has a certain probability. I think the issue is when thinking about it as a Superbowl win, instead of average change in weight or a more approach example.
    Stop cowardice, ban guns!

+ Reply to Thread
Page 1 of 2 1 2 LastLast

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats