area under the ROC curve for a cutoff point

[gunelpinar]

I kindly requested help about a study of mine concerning ROC curve analysis in medical diagnosis.

I use R software programme to find optimal cutoff point for a continuous variable.

Firstly, I generate a random continuous variable for a diseased and non-diseased subjects from normal distribution with different parameters, which provides an AUC of 0.80.
Then, I find the optimal cutoff point using "pROC" package and "roc()" function.
Then, I reclassify the original continuous variable according to the cutoff point that I found from "roc()" function.
And, finally I calculate the area under the ROC curve according to this new classification.
But, at every turn, I obtain the same AUC that is 1.00.

Why I obtain the value of 1.00 at every turn.

Thank you very much.

 

[hlsmith]

If your non-binary defined AUC was 0.8 and now you are getting 1.00, it seems you may be misspecifying the model or perhaps when you are turning the cutoff into the binary level you are defining it incorrectly.

Are you able to just run a 2x2 continency table for predictor and outcome? If so, this should show you what you are working with and you should be able to calculate the AUC using the A and D cells divided by the total of cells A, B, C, and D.

Let us know what you see with the contingency table.

 

[gunelpinar]

Thank you very much for your interest.

I would like to share R code for finding the optimal cutoff point using "pROC" package and "roc()" function.
And I also would like to share R code of the simulation which I conduct for comparing performance of ROC and Recursive Partitioning.

Could you please check and respond to me where I misspecifying?

Thank you very much.

R code for the simulation study:

sink("C:/deneme/yeni1.txt")

for (i in 1:500)

{


dat1<-data.frame(var=rnorm(50, 2.5, 0.25),group=rep(0, 50))
dat2<-data.frame(var=rnorm(50, 2.80, 0.25), group=rep(1, 50))
dat<-rbind(dat1, dat2)

y_roc<-roc(group ~ var, data=dat)
y_coords<-coords(y_roc, "b", "se")
y_rpart<-rpart(group ~ var, data=dat)
a<-summary(y_rpart)
opt<-which.min(y_rpart$cptable[,"xerror"])
cp<-y_rpart$cptable[opt, "CP"]
y_prune<-prune(y_rpart, cp=cp)
print(dat)
print(a)
print(y_prune)
print(y_coords)
}