Short version: You have a series of binary events, presumed independent but not known for sure. A subset of these series show all successes or all failures. If the subset was the entire dataset, it would be statistically improbable (which would force us to revisit the assumption of the underlying probability or the independence of events). But the subset is not the entire dataset. How do you calculate the probability of that subset occurring, for a given dataset size?
Long version (with my meandering attempts to figure this out myself, unsuccessfully):
I don't have a particular problem I am looking to solve, I am more interested in understanding how to handle a certain class of problems.
Let's say I flip a fair coin 7 times in a row and it comes up heads each time. Each flip is independent. It can be easily calculated that the probability of 7 heads in a row is about 0.8%.
It is also pretty straight forward, with binomial probability, to be able to look at a data set and see if a sample is consistent with previous assumptions. Let's say a free throw shooter makes 65 shots in 100 tries, and the prior probability for that free throw shooter was 50%. In this scenario, it's much more probable that the free throw shooter improved their shooting, than they just got lucky.*
Which brings me to the class of problem I'm trying to solve. The "hot hand" problem. The hot hand is when an athlete goes on a hot streak. In basketball, it may be a player making 8 consecutive field goals. The idea is as follows: the player makes a few in a row, and gets a boost of confidence, which makes the subsequent shots more likely to go in. A Lindy effect, if you will. Of course, this is called a fallacy for good reason--with hundreds of athletes across dozens of sports, someone is bound to be "hot" at any given time.
But... but! Can we truly say that basketball field goal attempts are truly independent? It's easy to speculate that some time sequences will be correlated. Did the athlete's newborn wake up every hour the previous night? Has a nagging injury finally healed? Did the coach rest the player the previous game, so they are fresh?
To put it in mathematical terms, let's say I have an event with an assumed 50% probability, let's say a free throw shooter in a controlled environment. Let's say we are unsure if the events are independent. Finally, let's say the shooter makes 7 shots in a row.
If the # of attempts is exactly 7, we would be forced to concluded that either the shooter has more than 50% odds of making a basket, or the events are dependent. But if the # of attempts is 100,000,000, making any 7 in a row is pretty meaningless.
So my question is how to approach that mathematically. One way I considered doing that, is that the chance of 7 in a row is assumed 0.8%. If we have 100 "7 shot attempts", the odds of getting at least 1 set of 7 in a row is about 45%. But how do I define what a "7 shot attempt" is? Here were a couple things I tried
The problem with the first definition, is the "7 shot attempts" are clearly not independent. For example, missing attempt #2 invalidates both "7 shot attempts". If you used this method, you would need 106 shots to get 100 "7 shot attempts." I simulated this with a monte carlo engine. With 106 shots of 50% probability, completely independent, the chances of 7 or more in a row was about 20%, well short of the 45% presumed above.
The second definition is equally problematic. It would ignore the fact that making 4 through 10 counts as 7 in a row. Again, it fails the monte carlo simulation. If I use 700 shots to get 100 "7 shot attempts", the odds of 7 or more in a row was about 75%, well above the 45%.
So both of those approaches are clearly incorrect, demonstrated both logically and through empirical simulations.
So here is my question - how do you handle this class of problem? How do you know if a sequence of successes or failures in a binary outcome is an outlier (assuming independence) or if it is statistically likely given the sample size.
*depending on how many 100 shot samples we have. If he does this every hour for years, eventually >= 65 will happen by luck alone.
Long version (with my meandering attempts to figure this out myself, unsuccessfully):
I don't have a particular problem I am looking to solve, I am more interested in understanding how to handle a certain class of problems.
Let's say I flip a fair coin 7 times in a row and it comes up heads each time. Each flip is independent. It can be easily calculated that the probability of 7 heads in a row is about 0.8%.
It is also pretty straight forward, with binomial probability, to be able to look at a data set and see if a sample is consistent with previous assumptions. Let's say a free throw shooter makes 65 shots in 100 tries, and the prior probability for that free throw shooter was 50%. In this scenario, it's much more probable that the free throw shooter improved their shooting, than they just got lucky.*
Which brings me to the class of problem I'm trying to solve. The "hot hand" problem. The hot hand is when an athlete goes on a hot streak. In basketball, it may be a player making 8 consecutive field goals. The idea is as follows: the player makes a few in a row, and gets a boost of confidence, which makes the subsequent shots more likely to go in. A Lindy effect, if you will. Of course, this is called a fallacy for good reason--with hundreds of athletes across dozens of sports, someone is bound to be "hot" at any given time.
But... but! Can we truly say that basketball field goal attempts are truly independent? It's easy to speculate that some time sequences will be correlated. Did the athlete's newborn wake up every hour the previous night? Has a nagging injury finally healed? Did the coach rest the player the previous game, so they are fresh?
To put it in mathematical terms, let's say I have an event with an assumed 50% probability, let's say a free throw shooter in a controlled environment. Let's say we are unsure if the events are independent. Finally, let's say the shooter makes 7 shots in a row.
If the # of attempts is exactly 7, we would be forced to concluded that either the shooter has more than 50% odds of making a basket, or the events are dependent. But if the # of attempts is 100,000,000, making any 7 in a row is pretty meaningless.
So my question is how to approach that mathematically. One way I considered doing that, is that the chance of 7 in a row is assumed 0.8%. If we have 100 "7 shot attempts", the odds of getting at least 1 set of 7 in a row is about 45%. But how do I define what a "7 shot attempt" is? Here were a couple things I tried
The problem with the first definition, is the "7 shot attempts" are clearly not independent. For example, missing attempt #2 invalidates both "7 shot attempts". If you used this method, you would need 106 shots to get 100 "7 shot attempts." I simulated this with a monte carlo engine. With 106 shots of 50% probability, completely independent, the chances of 7 or more in a row was about 20%, well short of the 45% presumed above.
The second definition is equally problematic. It would ignore the fact that making 4 through 10 counts as 7 in a row. Again, it fails the monte carlo simulation. If I use 700 shots to get 100 "7 shot attempts", the odds of 7 or more in a row was about 75%, well above the 45%.
So both of those approaches are clearly incorrect, demonstrated both logically and through empirical simulations.
So here is my question - how do you handle this class of problem? How do you know if a sequence of successes or failures in a binary outcome is an outlier (assuming independence) or if it is statistically likely given the sample size.
*depending on how many 100 shot samples we have. If he does this every hour for years, eventually >= 65 will happen by luck alone.
Last edited:
