+ Reply to Thread
Results 1 to 9 of 9

Thread: ROC cutoff value

  1. #1
    Points: 666, Level: 13
    Level completed: 32%, Points required for next Level: 34

    Posts
    11
    Thanks
    0
    Thanked 0 Times in 0 Posts

    ROC cutoff value




    Hello everyone
    While most of you is in vacation it came a cross my mind to post a question that I am trying to answer
    How to identify the cutoff value of your predictor variable on the ROC curve
    eg: id I am using the preoperative weight as a predictor for long term weight regain after surgery (dichotomous) and I need to estimate a value after which I can predict that the patient is most likely will not benefit from the proposed surgery at long term.

    Hw would I do this
    than you in advance

  2. #2
    Points: 666, Level: 13
    Level completed: 32%, Points required for next Level: 34

    Posts
    11
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: ROC cutoff value

    I knew that I can produce that table of coordinate variables but what value should I choose on what basis
    Thanks again

  3. #3
    Points: 666, Level: 13
    Level completed: 32%, Points required for next Level: 34

    Posts
    11
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: ROC cutoff value

    I am starting to feel from readings that it is a matter of choice am I correct

  4. #4
    Points: 666, Level: 13
    Level completed: 32%, Points required for next Level: 34

    Posts
    11
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: ROC cutoff value

    the other possible option that I identify the prevalence of success and set the cut off value for a sensitivity related to 1-prevalence of success.
    Am I on track or I am loosing it

  5. #5
    TS Contributor
    Points: 40,089, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Downloads
    gianmarco's Avatar
    Location
    Italy
    Posts
    1,367
    Thanks
    232
    Thanked 301 Times in 225 Posts

    Re: ROC cutoff value

    Hi!
    I am not an expert of Logistic Regression, and I have approached it just some months ago.
    I found useful this article by Peng-So, Logistic Regression Analysis and Reporting. A primer that is available HERE.

    May be that an optimal cut-off point can be derived from the ROC curve. But a useful (and more visually clear, at least to me) tool is the plot of specificity and sensitivity vs. verious cut-off points (see the article's figure 3). More info in the article, p. 48-49.

    I do not know many software that can plot such graph; MedCalc (commercial) can.
    I have recently discovered that in R (free) a number of packages are available for ROC analysis, but I have not tested them yet: ROCR and pROC seem interesting.

    Hope this helps,
    Regards
    Gm
    http://cainarchaeology.weebly.com/

  6. #6
    TS Contributor
    Points: 40,089, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Downloads
    gianmarco's Avatar
    Location
    Italy
    Posts
    1,367
    Thanks
    232
    Thanked 301 Times in 225 Posts

    Re: ROC cutoff value

    UpDate:

    I have just made some experiments with the R packages I was referring to in my earlier reply.
    I must confess that I found them a little tricky....Instead, I found the 'Epi' package quite user friendly, at least as far as plotting the ROC curve and "an" optimal cutoff value is concerned. it comes also with a detailed manual (LINK).

    If you are familiar with R, you could try this, just to give an example (using the dataset from the pROC package):
    Code: 
    library(pROC) #load the library
    data(aSAH) #load the dataset to be used with the Epi package
    library(Epi) #load the library
    ROC(form=outcome~s100b, data=aSAH) #compute and plot the ROC curve, using the variable outcome as y, s100b as x
    I found the output plot (attached) very useful and clean. It also summarizes the Logistic Regression parameters.

    Hope this helps
    Regards
    Gm
    Attached Images  
    http://cainarchaeology.weebly.com/

  7. #7
    Omega Contributor
    Points: 38,289, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,992
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: ROC cutoff value

    You have multiple options for the cutoff. Their are two typical prudent approaches. The first is Youden's Index - this maximizes the area under the curve. However, usually more appropriate is the first approach you listed, which would use your own selected cutoff based of the trade off of false negative and false positives. The importance of this latter more subjective method is that you as an educated clinician can decided if it is more important to have a lower false negative or false positive rate.

    An example I give in a lecture, is that a if the outcome may have dire consequences you may want to select the cutoff that errors on the side of more false positives than false negatives. In particular, a screening should ideally tell more people incorrectly that they may be positive for HIV than incorrectly telling them they are negative. Under this rationale, you have follow-up tests for the positives to confirm findings in lieu of telling people they may not be positive when they are and potentially introducing greater risk for virus spread in the population.
    Stop cowardice, ban guns!

  8. #8
    Omega Contributor
    Points: 38,289, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,992
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: ROC cutoff value

    GM,

    Can you confirm what the "ir.eta" represents.

    Sidenote, adding the n-value and prevalence to the graph can help the reader perform all superfluous calculations.
    Stop cowardice, ban guns!

  9. #9
    TS Contributor
    Points: 40,089, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Downloads
    gianmarco's Avatar
    Location
    Italy
    Posts
    1,367
    Thanks
    232
    Thanked 301 Times in 225 Posts

    Re: ROC cutoff value


    Can you confirm what the "ir.eta" represents.
    @hlsmith:
    As it appears digging through the 'Epi' package .pdf (p. 95), that value represents the
    “optimal cutpoint” (i.e. where sens+spec is maximal)
    It can be swithed on/off with the parameter MX in the ROC() command. The same holds for the report of the model summary, shown in the plot when MI=TRUE.
    See for example:
    Code: 
    ROC(form=outcome~s100b, data=aSAH, MX=TRUE, MI=TRUE)
    Regards,
    Gm
    http://cainarchaeology.weebly.com/

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats