Classification with high positive predictive value

[dchalmer]

I have a two-class classification problem. I would like to train a multivariate classifier with 100% positive predictive value. In other words, I want the model to completely avoid one of the classes. For this application a low-ish sensitivity is OK as long as PPV is ~100%.
Do you have any suggestions of good techniques to use? Thank you!

 

[Dason]

100%? That's asking a lot. Plus you typically can't say what you want to get on these types of measures (PPV, NPV, ...) before running the data. You can specify which measures you value the most (and here apparently you value PPV the most) but saying that you want 100% is quite a bit. That's saying that if you say the test is positive then you guarantee the condition is positive. You're not leaving any room for errors. Is that really what you're saying?

Note that these values completely depend on the quality of the predictors to predict.

 

[dchalmer]

Thanks for the response Dason,
Yep, you got it - when the model predicts positive, I need a (near) guarantee that it's right.
Currently I'm using a genetic algorithm to create a linear model which maximizes sensitivity under the constraint that PPV=100%. I then relax the sensitivity of the model to accommodate a user-specified margin - which creates some room for errors. This approach is working, but has several drawbacks. I wanted to see if you knew of any other techniques that might do the job?
Thanks again!

 

[Planet]

I have a two-class classification problem. I would like to train a multivariate classifier with 100% positive predictive value. In other words, I want the model to completely avoid one of the classes. For this application a low-ish sensitivity is OK as long as PPV is ~100%.
Do you have any suggestions of good techniques to use? Thank you!

I know an article that did precisely what you ask.

Dr. Harvey wanted to predict deep vein thrombosis (DVT) using d-dimer (DD) as an attribute, but he sought 100% correct prediction of all positive DVD cases. So he ordered observations by their d-d value, and located the d-d value beneath which there were no observations positive for DVD. He then assessed the statistical significance and effect strength of the resulting model using an exact non-parametric methodology known as “optimal data analysis” (ODA). The ODA statistical paradigm explicitly maximizes model accuracy (rather than variance or value of the likelihood function). Dr. Harvey’s article was selected by the American College of Physicians Journal Club, and read by most practicing internists in the US.

Here is the link to the article citation: http://stroke.ahajournals.org/content/27/9/1516.long

Here is the citation to the seminal introduction to the ODA paradigm (which comes with software for Windows), and is available in many academic libraries: http://www.amazon.com/Optimal-Data-Analysis-Guidebook-Software/dp/1557989818

And here is a blog about the optimal data analysis (ODA) statistical paradigm: http://odajournal.wordpress.com/