+ Reply to Thread
Results 1 to 3 of 3

Thread: ROC Curve - Determining cut-off sensitivity and specificity

  1. #1
    Points: 4,824, Level: 44
    Level completed: 37%, Points required for next Level: 126

    Posts
    41
    Thanks
    19
    Thanked 1 Time in 1 Post

    ROC Curve - Determining cut-off sensitivity and specificity




    Hi All,

    I am using SPSS to run a ROC curve to determine the sensitivity and specificity of a psychological measure in detecting participants with and without a disorder.

    In the output, there is a 'Coordinates of the curve' table, and I was hoping to get some advice on how best to interpret it (I have attached an example table similar to my results). You will see that in the "positive if greater than or equal to" column the scores are 1.5, 2.5, etc.

    I need to round these scores to the nearest whole number (i.e., 1, 2), and was wondering if, for example, using a cut-off of 2.5 would that mean that those who score 3 or greater are classified as positive? Or is it 2 or greater?

    Hope that makes sense and I really appreciate all the support from this forum!
    Attached Files

  2. #2
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: ROC Curve - Determining cut-off sensitivity and specificity

    I did not open your file, but the typically scenario is that you calculate the SEN and SPEC for every value in your dataset as the cutoff (which is what the ROC curve is a presentation of). There are multiple ways of selecting the final cut-off threshold. The traditional method is using the Youden Index, which selects the largest area under the curve and treats over- and under--classifications the same. Another option is Gini index, which tries to minimize impurities in classifications. There is also the option where you select the cut-off based on clinical gestalt or desires - meaning you don't want the best balance, you want to have the fewest false negatives.


    Ideally if you had enough data, you would create the cut-off rule based on a random training set (e.g., 60% data) and then test it on the hold-out set (e.g., 40% data).


    As for should you split it at 2.5 or 2, etc. If you are unsure, you can just calculate the SEN and SPEC for both of the values and select the one which bode best for your agenda.


    Let us know if you have any other questions.
    Stop cowardice, ban guns!

  3. The Following User Says Thank You to hlsmith For This Useful Post:

    ondansetron (06-01-2017)

  4. #3
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: ROC Curve - Determining cut-off sensitivity and specificity


    Given your gold standard, what is the prevalence of the disorder in your dataset?
    Stop cowardice, ban guns!

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats