Ranking Method Comparison

#1
Hi,

I have 15 items that need to be ranked from 1 to 15.
I have two methods to do the ranking, method A and method B.

Method A is where I have 15 users rank all items on a scale of 1 to 5. (5 Being best).
I then take the sum of user ranks for a given item and the highest sum becomes the rank 1 item, and the lowest sum becomes the rank 15 item.

Question 1) Is this a useful method of ranking items? Is there anything wrong in my approach?

Method A is now considered my benchmark ranking.

Method B is another algorithm that ranks the content from 1 to 15.

My hypothesis is that Method B is viable method and can be used because it yields similar results to benchmark and more traditionally accepted Method A.

Question 2) How do you compare the ranking methods? Should I just compare the distance between ranking and plot a graph? Is there a more meaningful method?
Friedman Test? T-Test?

My data will look something like this:


ITEM | Rank - Method A | Rank - Method B
01 | Rank-a | Rank-b
02 | Rank-a | Rank-b
03 | Rank-a | Rank-b
04 | Rank-a | Rank-b
05 | Rank-a | Rank-b
06 | Rank-a | Rank-b
07 | Rank-a | Rank-b
08 | Rank-a | Rank-b
09 | Rank-a | Rank-b
10 | Rank-a | Rank-b
11 | Rank-a | Rank-b
12 | Rank-a | Rank-b
13 | Rank-a | Rank-b
14 | Rank-a | Rank-b
15 | Rank-a | Rank-b

The idea is to show that Rank-b is same or close to Rank-a thus showing that Method B is same/similar to Method A. What is the best way to do so?
 
Last edited: