# Thread: What is the difference between classical and bayesian understanding of probability?

1. ## What is the difference between classical and bayesian understanding of probability?

I am reading a book on monte carlo analysis and the author stresses in passing that the definition of probability used in an example in the book is the classical aka frequentist one, and that Bayesian understanding of probability is different. They say the difference is too complex to explain in the book.

The comment goes "Thus, we can define probability in terms of the relative frequency with which an event will occur if we repeated our "experiment" [a coin flip] a great many times. This is a classic "frequentist" interpretation of probability that lies at the heart of traditional or "classical" statistics." [they then to go on to point out that Bayesian's have a different interpretation of probability, but no details are provided].

I was hoping for a brief explanation of how the two differ.

2. ## Re: What is the difference between classical and bayesian understanding of probabilit

Welp there is a metric ****-ton of articles about the difference, so it'd be worth reading up, but a very simple definition would be that a frequentist interprets probability just how it's stated in your quote:

the relative frequency with which an event will occur if we repeated our "experiment" [a coin flip] a great many times.
This means that you can only usefully talk about the probability of events happening if you can at least imagine in principle conducting repeated "trials" (experiments, coin flips, whatever) in which sometimes the event takes place and sometimes it doesn't. To a frequentist, it's meaningless to ask what the probability it is that some hypothesis is true; the hypothesis is either true or it isn't (it isn't true in some trials and false in others). This is part of the reason why the probabilities we talk about in frequentism (e.g., p values) refer to how probable it is that a particular thing will be observed in an experiment, not how probable hypotheses themselves are. We can imagine conducting repeated experiments in which sometimes a thing happens and sometimes it doesn't. (But the hypothesis will simply remain either true or false).

In the Bayesian interpretation of probability, a probability is a state of certainty or belief with respect to some proposition. That probability could refer to the rate at which an event (e.g., a coin flip coming up heads) occurs over repeated trials, but it doesn't have to: A Bayesian can also refer to the probability that a particular hypothesis is true.

And Bayes theorem shows us how to combine existing knowledge/beliefs/information along with the data observed to make statements about probabilities.

(Note: That's a super simplistic version!)

3. ## The Following User Says Thank You to CowboyBear For This Useful Post:

Buckeye (02-07-2017)

4. ## Re: What is the difference between classical and bayesian understanding of probabilit

As a non-statistician I guess the frequentist position makes more sense to me. The sky is, to be simplistic, either blue or its not. That is hypothesis are either true or they are not since to me the hypothesis is about what is true in the population not the sample (at a given time anyhow obviously reality changes in some cases).

5. ## Re: What is the difference between classical and bayesian understanding of probabilit

Originally Posted by noetsi
As a non-statistician I guess the frequentist position makes more sense to me. The sky is, to be simplistic, either blue or its not. That is hypothesis are either true or they are not since to me the hypothesis is about what is true in the population not the sample (at a given time anyhow obviously reality changes in some cases).
Just to be clear: The Bayesian position isn't that a particular hypothesis is sometimes true and sometimes false. The Bayesian position is that we can make probability statements about things that are either true or false. So a Bayesian can say something like "there is a 95% probability that my hypothesis is true", whereas a frequentist can't.

Anyway the best thing to do if you're interested in the difference between the two perspectives is to read more. Some sources to start with:

Kruschke, J. K. (2010). What to believe: Bayesian methods for data analysis. Trends in Cognitive Sciences, 14(7), 293–300. https://doi.org/10.1016/j.tics.2010.05.001
Gigerenzer, G., Krauss, S., & Vitouch, O. (2004). The null ritual: What you always wanted to know about signiﬁcance testing but were afraid to ask. In D. Kaplan (Ed.), The Sage handbook of quantitative methodology for the social sciences (pp. 391–408). Thousand Oaks, CA: Sage.
Dienes, Z. (2011). Bayesian versus orthodox statistics: Which side are you on? Perspectives on Psychological Science, 6(3), 274–290. https://doi.org/10.1177/1745691611406920

6. ## The Following User Says Thank You to CowboyBear For This Useful Post:

noetsi (02-07-2017)

7. ## Re: What is the difference between classical and bayesian understanding of probabilit

I have a follow up question. Because conditional probabilities and Bayes' Theorem piqued my interest as an undergraduate.

Are these two definitions of probability always used together? In other words, is it possible for someone to think entirely in terms of Bayesian statistics and "ignore" the frequentist perspective? I don't imagine so. But I'm curious.

I really enjoy what I've seen of Bayesian statistics.

8. ## Re: What is the difference between classical and bayesian understanding of probabilit

One of the reasons a lot of people like the bayesian perspective is entirely because they prefer the bayesian interpretation of probability.

Want to talk about the probability your favorite team will win the super bowl? Well try thinking about what a frequentist defines probability as and think if it makes sense to talk about the probability that the Packers (obviously this is everybody's favorite team) wins the super bowl next year. With a bayesian interpretation it's very natural to talk about probability of one time events like this. If you have this chat with a frequentist there is a non-zero probability (hah) that you'll hear something about multiple universes or hypothetical timelines and it starts to sound *really complicated* just to talk about how you can define probability for an event like that.

9. ## Re: What is the difference between classical and bayesian understanding of probabilit

Originally Posted by Buckeye
I have a follow up question. Because conditional probabilities and Bayes' Theorem piqued my interest as an undergraduate.

Are these two definitions of probability always used together? In other words, is it possible for someone to think entirely in terms of Bayesian statistics and "ignore" the frequentist perspective? I don't imagine so. But I'm curious.

I really enjoy what I've seen of Bayesian statistics.
There are a few issues here. First note that if you're just using bayes theorem to do a probability calculation that doesn't mean you're doing bayesian statistics.

Also it's entirely possible to be a frequentist and use bayesian methods. You might just be interested in different estimators that have nice properties and want to discuss the frequentist properties of such estimators. It's also possible to be a bayesian and and want to explore the frequentist properties of your estimators. They aren't entirely mutually exclusive schools of thought.

10. ## The Following User Says Thank You to Dason For This Useful Post:

Buckeye (02-07-2017)

11. ## Re: What is the difference between classical and bayesian understanding of probabilit

Okay, I have a limited knowledge under my belt. So, I naturally attributed Bayes' Theorem to that school of thought. As a side note, I'm excited to pursue a master's in the fall. Applications of statistics are endless!

12. ## Re: What is the difference between classical and bayesian understanding of probabilit

While this may not be theoretically the case I commonly read analysis that mix bayseian and classical statistics. That is they use Bayesian approaches sometimes and frequentist statistics other times.

13. ## Re: What is the difference between classical and bayesian understanding of probabilit

Originally Posted by Dason
If you have this chat with a frequentist there is a non-zero probability (hah) that you'll hear something about multiple universes or hypothetical timelines and it starts to sound *really complicated* just to talk about how you can define probability for an event like that.
God**** frequentists with their multiverses and superpopulations...

14. ## Re: What is the difference between classical and bayesian understanding of probabilit

Originally Posted by Dason
If you have this chat with a frequentist there is a non-zero probability (hah) that you'll hear something about multiple universes or hypothetical timelines and it starts to sound *really complicated* just to talk about how you can define probability for an event like that.
Are you half-kidding or is this for real? As in, does the frequentist interpretation of probability implicitly assumes the need for multiple universes/timelines? Is it something you heard or is it something that someone has tried to justify somewhere like in a journal article or book or something?

15. ## Re: What is the difference between classical and bayesian understanding of probabilit

Originally Posted by spunky
Are you half-kidding or is this for real? As in, does the frequentist interpretation of probability implicitly assumes the need for multiple universes/timelines? Is it something you heard or is it something that someone has tried to justify somewhere like in a journal article or book or something?
Dunno about Dason, but I'm pretty sure I've seen this kind of thing. I mean, after all the winner of next year's superbowl isn't a random variable: You can't say it's 65% likely to be the Packers in the frequentist definition, because there aren't going to be multiple 2018 superbowls in which sometimes the Packers win and sometimes they don't. So to make a frequentist probability statement you have to make come up with some imaginary way in which there are multiple "trials". (Not that the frequentist interpretation implicitly requires multiverses - it just kinda requires them if we want to stretch frequentist probability into making statements about things that it's not designed to make statements about).

16. ## The Following User Says Thank You to CowboyBear For This Useful Post:

Dason (02-07-2017)

17. ## Re: What is the difference between classical and bayesian understanding of probabilit

The simplest way to explain the randomness in quantum physics is the multiple universe interpretation of Everett - so frequentists might in the end be right after all.

regards

18. ## Re: What is the difference between classical and bayesian understanding of probabilit

Originally Posted by noetsi
As a non-statistician I guess the frequentist position makes more sense to me.
I am the least qualified person who can talk about theoretical aspects of statistics in general, and of Bayesian approaches in particular. I wish to just stress that Bayesian "standpoint" has its relevance in those fields where you want to assess, for instance, how likely is your prior knowledge on the basis of what you observe, i.e. data.

For instance, in archaeological radiocarbon dating, Bayesian statistic provides scientists with a formal way to update one's belief in the light of the data (s)he has currently in his/her hand. For example, we got radiocarbon data from context A, B, and C, and we have a prior belief/hypothesis (deriving from other independent observations) as to the fact that A is older than B, and B is older than C. Bayesian analysis allows us to arrive at the posterior probability as to the soundness of our prior hypothesis. In other words, given our prior hypothesis AND given the data, we can "update" our prior belief in the light of what data are telling us (so getting a posterior probability).

I found a book whose first sections are good for making non-stat people (like me) grasp some basic principle of Bayesian inference:
https://www.amazon.com/Bayes-Rule-Tu.../dp/0956372848

gm

19. ## The Following User Says Thank You to gianmarco For This Useful Post:

Buckeye (02-08-2017)

20. ## Re: What is the difference between classical and bayesian understanding of probabilit

Right on Rogojel it is all about M-verses. String theory is so 90's for buzz terms. I am also interested in the idea of M-verses. Opens the doors to many awesome movies as well. And I keep wondering if our universe is ever expanding cant we stave off our own freezing demise with some type of human generated nuclear power. I digress, but was thinking the same thing about M-verses.

I don't get all gummed up by the idea that given the event in a super-population is true, that realization has a certain probability. I think the issue is when thinking about it as a Superbowl win, instead of average change in weight or a more approach example.