ROC curves and attitudes dilemma

#1
Dear contributors,

I am conducting a study looking at attitudes of a group to the use of music as a form of therapy for people with dementia. I want to see if they are positive, neutral or negative (at least negative or positive) in my sample (N=92). My hypothesis based on previous studies is that the majority of the group will have positive attitudes.
In order to do that, I have constructed a scale and done the usual reliability and validity checks. The scale (SAMD) ranges from 19 to 95 (continuous), with highest scores reflecting more positive attitudes.
I am stuck as to how to classify negative attitudes, neutral and positive attitudes (at least negative or positive). My supervisor suggested ROC but I cannot see for the life of me how that makes any sense. (Correct me if I'm wrong) ROC curves are used to determine cut off scores - he is presuming that a cut off score of let's say 50 (which I am supposed to calculate so that it has the best specificity and sensitivity) would determine negative from positive attitudes. But when it comes down to it, I still don't know who has positive and who has negative attitudes so dichotomise the variable SAM-D.

I chose to split the distribution of the range of scores into three sections and categorized negative, neutral and positive attitudes accordingly (frequency statistics for the 33rd and the 66th percentile ) but that seems ad-hoc.

I have also administered two other scales (continuous variables as well) to the sample alongside them, one looking at attitudes to dementia and another looking at stress. My supervisor suggests that I do ROC with the attitudes to dementia scale (ADQ) and the SAMD.

I am trying to work out if ROC is the best way to find if my sample has positive attitudes. If so, I would appreciate it if someone explained it to me! If ROC is not the way, I would be grateful of any other methods.

Many thanks in advance
Almitra
 

hlsmith

Less is more. Stay pure. Stay poor.
#2
So you used a scale to evaluate if a person liked music, but the scale has no cut-offs? What have people used in the past, I would imagine at some point somebody had to describe how the scale works.

Well you cannot use the ROC curve and the SAMD to find cut-offs unless you know the true opinion of the individuals typically based on a gold standard. So if I was going to use a continous lab value to define disease, I need to know the actual disease status in the study population or have a really good proxy. Let me know if you have questions.
 
#4
Hello, I made the scale myself.
I completely understand that unless I know the true opinion of the individuals I cannot make cut offs.
Previous attitude scales speak of no cut offs and use the scales qualitatively (eg 76% agreed with "When I sing in the bathroom I get shouted at") -
It is one thing using my results qualitatively, and another thing having to reject a null hypothesis statistically for your thesis results, in this case the Ho being 'staff have negative attitudes to music'.
Any suggestions on alternatives?
Many thanks, I am very grateful for your help!!!
Almitra
 

hlsmith

Less is more. Stay pure. Stay poor.
#5
Can you state how the question was introduced or phrased to the individuals? Was it just, on a scale for 1-100 how much did you like the music?

"I have constructed a scale and done the usual reliability and validity checks." What reliability and validity checks did you actually perform?
Reliability: if asked to repeat got same response
Validity: compared to standard or truth got comparable results

"he scale (SAMD) ranges from 19 to 95 (continuous)" what does SAMD stand for and was the 19-95 not the range but the range of responses?
 
#6
Thanks for the reply.
I am investigating if dementia staff have positive attitudes to the use of music for their patients as a therapy. For this reason I made a 19 item scale, the scores of which range from 15 (minimum) to 95 (maximum on the scale). SAM-D stands for 'Staff Attitudes to Music - Dementia'. It is a 5-point Likert scale ranging from agree (5 points) to disagree (5 points). So the highest someone can score if 5x19=95.

Reliability - I did exploratory and confirmatory factor analysis and calculated cronbach's alpha for each subscale (3 factors emerged). could not check test-retest reliability
Validity - established through thorough pilot work, expert review.

Hope this helps.
Almitra
 

hlsmith

Less is more. Stay pure. Stay poor.
#8
Well, do you have a subset of questions (construct) that focuses on if they like the music. If you specifically ask it, I don't know if you need to create a score for the instrument as a whole.

If you did not design the instrument to be provide an overall score toward liking music, it may be difficult to now make is a culmination score toward music, unless you have the goldstandard data will discussed before.