+ Reply to Thread
Results 1 to 3 of 3

Thread: area under the ROC curve for a cutoff point

  1. #1
    Points: 55, Level: 1
    Level completed: 10%, Points required for next Level: 45

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Post area under the ROC curve for a cutoff point




    I kindly requested help about a study of mine concerning ROC curve analysis in medical diagnosis.

    I use R software programme to find optimal cutoff point for a continuous variable.

    Firstly, I generate a random continuous variable for a diseased and non-diseased subjects from normal distribution with different parameters, which provides an AUC of 0.80.
    Then, I find the optimal cutoff point using "pROC" package and "roc()" function.
    Then, I reclassify the original continuous variable according to the cutoff point that I found from "roc()" function.
    And, finally I calculate the area under the ROC curve according to this new classification.
    But, at every turn, I obtain the same AUC that is 1.00.

    Why I obtain the value of 1.00 at every turn.

    Thank you very much.

  2. #2
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: area under the ROC curve for a cutoff point

    If your non-binary defined AUC was 0.8 and now you are getting 1.00, it seems you may be misspecifying the model or perhaps when you are turning the cutoff into the binary level you are defining it incorrectly.

    Are you able to just run a 2x2 continency table for predictor and outcome? If so, this should show you what you are working with and you should be able to calculate the AUC using the A and D cells divided by the total of cells A, B, C, and D.

    Let us know what you see with the contingency table.
    Stop cowardice, ban guns!

  3. #3
    Points: 55, Level: 1
    Level completed: 10%, Points required for next Level: 45

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: area under the ROC curve for a cutoff point


    Thank you very much for your interest.

    I would like to share R code for finding the optimal cutoff point using "pROC" package and "roc()" function.
    And I also would like to share R code of the simulation which I conduct for comparing performance of ROC and Recursive Partitioning.

    Could you please check and respond to me where I misspecifying?

    Thank you very much.

    R code for the simulation study:

    sink("C:/deneme/yeni1.txt")

    for (i in 1:500)

    {


    dat1<-data.frame(var=rnorm(50, 2.5, 0.25),group=rep(0, 50))
    dat2<-data.frame(var=rnorm(50, 2.80, 0.25), group=rep(1, 50))
    dat<-rbind(dat1, dat2)

    y_roc<-roc(group ~ var, data=dat)
    y_coords<-coords(y_roc, "b", "se")
    y_rpart<-rpart(group ~ var, data=dat)
    a<-summary(y_rpart)
    opt<-which.min(y_rpart$cptable[,"xerror"])
    cp<-y_rpart$cptable[opt, "CP"]
    y_prune<-prune(y_rpart, cp=cp)
    print(dat)
    print(a)
    print(y_prune)
    print(y_coords)
    }

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats