Cohen's Kappa Chance Calculation

#1
I am confused about the "total agreement by chance" calculation in Cohen's Kappa statistic.

Say that Rater 1 agrees in 13/19 of the cases, and Rater 2 agrees in 14/19 of the cases. Why is the calculation 13/19 * 14/19. That amount is then added to the product of the total non agreement observations in a similar fashion to get the "total agreement by chance."

Seems like something is lurking behind the scenes I am not quite understanding. Are there any sort of basic probability rules I am missing or that are at play here? I am just not sensing how "chance" agreement is found from doing these simple calculations on what was actually observed.

I know the chance agreement is only a part of the final Kappa statistic...it is just the chance calculation that confuses me.
 
Last edited:

rogojel

TS Contributor
#2
Hi,
imagine that both raters toss a coin and rate Passed if it's heads. 0.5*0.5 times they will agree, just by accident.

regards