Hi,

I have 15 items that need to be ranked from 1 to 15.

I have two methods to do the ranking, method A and method B.

Method A is where I have 15 users rank all items on a scale of 1 to 5. (5 Being best).

I then take the sum of user ranks for a given item and the highest sum becomes the rank 1 item, and the lowest sum becomes the rank 15 item.

Question 1) Is this a useful method of ranking items? Is there anything wrong in my approach?

Method A is now considered my benchmark ranking.

Method B is another algorithm that ranks the content from 1 to 15.

My hypothesis is that Method B is viable method and can be used because it yields similar results to benchmark and more traditionally accepted Method A.

Question 2) How do you compare the ranking methods? Should I just compare the distance between ranking and plot a graph? Is there a more meaningful method?

Friedman Test? T-Test?

My data will look something like this:

ITEM | Rank - Method A | Rank - Method B

01 | Rank-a | Rank-b

02 | Rank-a | Rank-b

03 | Rank-a | Rank-b

04 | Rank-a | Rank-b

05 | Rank-a | Rank-b

06 | Rank-a | Rank-b

07 | Rank-a | Rank-b

08 | Rank-a | Rank-b

09 | Rank-a | Rank-b

10 | Rank-a | Rank-b

11 | Rank-a | Rank-b

12 | Rank-a | Rank-b

13 | Rank-a | Rank-b

14 | Rank-a | Rank-b

15 | Rank-a | Rank-b

The idea is to show that Rank-b is same or close to Rank-a thus showing that Method B is same/similar to Method A. What is the best way to do so?

I have 15 items that need to be ranked from 1 to 15.

I have two methods to do the ranking, method A and method B.

Method A is where I have 15 users rank all items on a scale of 1 to 5. (5 Being best).

I then take the sum of user ranks for a given item and the highest sum becomes the rank 1 item, and the lowest sum becomes the rank 15 item.

Question 1) Is this a useful method of ranking items? Is there anything wrong in my approach?

Method A is now considered my benchmark ranking.

Method B is another algorithm that ranks the content from 1 to 15.

My hypothesis is that Method B is viable method and can be used because it yields similar results to benchmark and more traditionally accepted Method A.

Question 2) How do you compare the ranking methods? Should I just compare the distance between ranking and plot a graph? Is there a more meaningful method?

Friedman Test? T-Test?

My data will look something like this:

ITEM | Rank - Method A | Rank - Method B

01 | Rank-a | Rank-b

02 | Rank-a | Rank-b

03 | Rank-a | Rank-b

04 | Rank-a | Rank-b

05 | Rank-a | Rank-b

06 | Rank-a | Rank-b

07 | Rank-a | Rank-b

08 | Rank-a | Rank-b

09 | Rank-a | Rank-b

10 | Rank-a | Rank-b

11 | Rank-a | Rank-b

12 | Rank-a | Rank-b

13 | Rank-a | Rank-b

14 | Rank-a | Rank-b

15 | Rank-a | Rank-b

The idea is to show that Rank-b is same or close to Rank-a thus showing that Method B is same/similar to Method A. What is the best way to do so?

Last edited: