# How to compare multiple testers wine ratings?

#### achalk

##### New Member

Problem description: I have 20 wines and 6 tasters. Each taster ranks the wines from 1 to 20.

Question: How do I test the hypothesis that the taster's rankings were the same?

Many thanks!

#### CowboyBear

##### Super Moderator
Can you explain what your hypothesis is in a bit more detail? Do you mean that you hypothesise that the tasters will rank all the wines in exactly the same order?

#### achalk

##### New Member
Correct. That is what I would use as the null hypothesis.

#### CowboyBear

##### Super Moderator
If there were any differences in the rankings, then that null hypothesis is false.

#### achalk

##### New Member
Right. So back to my question: How do I conduct a statistical test of that hypothesis?

#### CowboyBear

##### Super Moderator
You open up your data on your computer screen. You look at the rankings. Are they exactly the same for all the raters?

#### CowboyBear

##### Super Moderator
Achalk, I'm not "bullying" you as you claim in your report. The point I am trying to guide you to is that you don't need any sophisticated statistical test to reject the (implausible) null hypothesis that you've specified. Quite literally, if you look at the rankings, and some of the tasters ranked the wines differently from others, that's enough to reject the null hypothesis you've chosen.

The bigger point here is that a useful statistical analysis depends heavily on what specific questions or hypotheses you decide to investigate. Forgetting about statistical jargon and null hypotheses, what are you actually trying to find out here?

#### rogojel

##### TS Contributor
hi,
you could look at Cohens Kappa , and at Appraiser Agreement Analysis.
regards
Sandor

#### achalk

##### New Member
Many thanks, rojojel. I will follow up on both of those.