+ Reply to Thread
Results 1 to 3 of 3

Thread: How to assess a new diagnostic test apart from ROC

  1. #1
    Points: 1,364, Level: 20
    Level completed: 64%, Points required for next Level: 36

    Posts
    8
    Thanks
    4
    Thanked 0 Times in 0 Posts

    How to assess a new diagnostic test apart from ROC




    I have a diagnostic scoring system that uses 6 variables, each an ordinal or dichotomous value (some 0 to 1, others 0 to 4) (that when summed together are used to predict a dichotomous outcome (ie presence or absence of a disease). I have performed a ROC analysis and the results are good. The summary score has a very high area under the curve (AUC) of 0.93. The ROC analysis of each individual score has a lower AUC. I also thought it would be appropriate to perform a principal components analysis but the Kaiser-Meyer-Olkin measure of sampling adequacy was insignificant (about 0.4) and so I had to abandon that. Does this approach of doing a ROC and leaving it that seem like a reasonable approach to testing this new diagnostic score? Should I employ any other statistical tests to test the data? I have run out of ideas.... Many thanks in advance....

  2. #2
    Omega Contributor
    Points: 38,396, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    7,001
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: How to assess a new diagnostic test apart from ROC

    The ROC curve is great, but you should always also present the prevalence, SEN, SPEC, PPV, and NPV - so that reviewers can understand where the diagnostic falters (false positives and false negatives). Also, don't forget to incoporate confidence intervals around all of your values.

    Lastly, another option (perhaps) may be field testing the approach on different samples. If you have developed it with one sample you also what to make sure it functions well on all applicable patients.
    Stop cowardice, ban guns!

  3. The Following User Says Thank You to hlsmith For This Useful Post:

    TS1 (06-14-2013)

  4. #3
    Points: 1,364, Level: 20
    Level completed: 64%, Points required for next Level: 36

    Posts
    8
    Thanks
    4
    Thanked 0 Times in 0 Posts

    Re: How to assess a new diagnostic test apart from ROC


    That's very useful. i will put those other measures in. I will also plan to validate it in the way you mention. Many thanks for your reply

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats