Reducing likert-type scales from 10 point scale to lower order point scal

#1
Have made lots of movement in looking at this project...

Overall Aim:
To construct a clinical medical image scales (10 images in scale)
Objectives:
  1. To construct a 10 point scale using responses from multiple observers
  2. Use scale to find areas of confusion between observers
  3. Using areas of confusion - reduce the 10 point scale to a lower point scale

Overview: To devise 10 point clinical grading scales. Images are labelled 1 to 10. Images 1 & 10 are fixed images. Observer number =11, ranking 8 images (2-9). Using this data I am use these expert opinions to devise a 5 point scale.
So far have conducted: Spearman's between each observer, Kendall's Concordance for multi rater ranking. Plotted visual data. Use Psychometric techniques for error scoring to look at error scores compared to the mean scores.

The point I'm at now is to how to reduce / collapse this scale.. Identified inter-observer 'confusion areas' to see where images have been ranked differently.


So here is my raw data for one of the scales. 11 experts (col1) v 8 rankings (fixed end-points not given below) . From the below Obs= Observer Initials. Top Row is the 'Image rank.' This is one of the simpler multi-rater scales I have. Where in Positions 2,3 & 9 - all observers choose the images labelled 2,3 and 9, perfect agreement. However Images 4,5 & 6 are all placed in the middle of the scale but are ordered differently. Similarly in positions 7/8.


How do 'prune'/ collapse/ reduce this 10 point scale a 7 point scale where:


Points 1& 7 are fixed and are not included below
Point 2 remains point 2
Point 3 remains point 3
Point 4 merges columns 4-6
Point 5 merges columns 7-8
Point 6 is from old point


Obs 2 3 4 5 6 7 8 9
PG 2 3 5 4 6 8 7 9
DH 2 3 4 6 5 8 7 9
RS 2 3 4 5 6 7 8 9
RSt 2 3 4 6 5 7 8 9
VS 2 3 4 5 6 7 8 9
CF 2 3 4 5 6 7 8 9
MB 2 3 4 6 5 7 8 9
YFW 2 3 4 5 6 7 8 9
JM 2 3 6 4 5 7 8 9
MA 2 3 6 4 5 7 8 9
WN 2 3 5 4 6 8 7 9

As i say is one of the more straightforward sets of data where some of the response are just swapped around. The set below is a little more complicated :confused:

Obs 2 3 4 5 6 7 8 9
Obs 2 3 4 5 6 7 8 9
PG 2 3 4 6 7 5 8 9
DH 2 3 4 5 6 7 8 9
RS 3 2 4 6 5 7 8 9
RSt 3 2 4 5 6 7 8 9
VS 4 3 6 2 5 7 9 8
CF 2 3 4 5 7 6 8 9
MB 2 3 6 4 7 5 8 9
YFW 2 3 6 4 8 7 5 9
JM 3 2 4 5 7 6 9 8
MA 4 2 6 3 5 7 8 9
WN 4 2 3 7 6 5 8 9

I want to reduce the scale the recalc Spearman, Kendall Concordance etc to show the improved values of theoretically pruning.

Have come a loooooonnnggg way with looking at this task and was trying to stretch myself so I could look at other techniques. Is this like reducing the results of likert scales and then using this to prune the 'points' in the scale to a smaller number. Not sure if I am just over-saturating my self with too many ideas: ANN's pruning, likerts, decision trees etc.

Any help greatly appreciated :D:wave:
 
Last edited: