# How to calculate inter-observer/ inter-rater reliability of ordinal data in R?

#### Manatis

##### New Member
Hello,

I have the question which measurement to use in order to calculate the inter-observer reliability in R once for an ordinal and once for a nominal data set.

Part1) Ordinal data set
2 observers gave scores on a scale from 1.0 to 5.0 with steps of 0.25
Example of data:
observ1 <-(c=(2.25, 3.0,2.75, 3.5,3.5))
observ2<-(c=(2.5,3.0,2.75,3.25,3.5))

a) Is Cohen´s Kappa a valid measurement for this type of data sets? If yes, how can I calculate this with R? I have tried cohen.kappa {psych} without success, and am quite confused about the existence of more kappas in the packages irr, vcd and even base. Which one do you recommend?

b) In case Cohen´s Kappa is not feasible: Which measurement do you recommend and again: where is it to be found in R?

c) To make it more tricky, I wanted to compare the measurement (e.g. Kappa) using the data set given, to the (improved) repeatability measure from a second run, which accepts a deviation of 0.25 between observers. How do I incorporate this second run in R?

Part 2) Nominal data set
2 observers, example data sets:
Observer1=(c=(11,8,12,10))
Observer2=(c=(10,8,12,10))
1) Which measurement for the inter-observer reliability do you recommend and how do I calculate it with R?