ROC CURVE · AUC WITH DELONG INTERVAL · DELONG PAIRED TEST · YOUDEN CUT-OFF · PARTIAL AUC · DECISION CURVE · RUNS IN YOUR BROWSER · NOTHING UPLOADED

How well does this score tell cases from non-cases, and is it better than that one?

Paste one row per person or item with the outcome (disease or not, fraud or not) and one or more scores measured on the same rows: a lab marker, a questionnaire, a model's probability. You get ROC curves, each AUC with DeLong's interval, DeLong's paired test for every pair, cut-offs by Youden's J or a target sensitivity with predictive values and likelihood ratios, the partial AUC at high specificity and, for probabilities, a decision curve.

Nothing you enter leaves this browser tab. Statistical output from data you supply, not advice. Not medical, legal, tax, or financial advice.

Your data

Rows with a missing outcome or score are skipped and counted, so every score is compared on the same rows.

Scores to analyse (up to 8), and which way each points

Paste data to choose scores.

Method and citation

The curve and the AUC. Each distinct score value is a cut-off ("a case when the score is at least t"); the curve joins the sensitivity and false positive rate at each. The AUC is the probability that a random case scores higher than a random non-case, ties counted one half, which equals the area under that polyline (Hanley & McNeil 1982).

DeLong's interval and paired test. Every case gets a placement value, the share of non-cases it outscores, and every non-case the share of cases that outscore it. The AUC's variance is the variance of the case placements over the number of cases plus that of the non-case placements over the number of non-cases. For several scores on the same rows the same placements give a covariance matrix, and the difference of two AUCs is tested with its own standard error, which accounts for the correlation (DeLong, DeLong & Clarke-Pearson 1988); with three or more scores a chi-square test asks whether all are equal. Placements are computed from midranks (Sun & Xu 2014), so 200,000 rows take a fraction of a second.

Cut-offs and accuracy. Youden's (1950) J = sensitivity + specificity − 1; the closest point to the top-left corner; the most specific cut-off with sensitivity at or above the target, and the most sensitive with specificity at or above it. At a cut-off: sensitivity, specificity and predictive values with Wilson (1927) intervals (predictive values at a prevalence you give are by Bayes' rule, without an interval), and likelihood ratios with intervals on the log scale (Simel, Samsa & Matchar 1991).

Partial AUC and decision curves. The area over false positive rates 0 to e, standardised to a 0.5 (useless) to 1 (perfect) scale by McClish (1989), with a stratified bootstrap interval. For a score that is a predicted probability, net benefit TP/N − FP/N × pt/(1 − pt) against treating everyone and no one (Vickers & Elkin 2006).

Verified by tests in the source against an independent computation (eval/reference.py): on the example, a tied rating scale and a simulated three-score case, the AUCs match scipy 1.18.1's Mann-Whitney U to 10⁻¹²; the DeLong covariance matrices match a direct O(mn) computation from the definition with numpy 2.5.3 to 10⁻¹⁴; the paired z, p-values and chi-square match scipy; the Wilson intervals match scipy.stats.binomtest; and on Hanley & McNeil's (1982) 109 rated CT images the AUC is 0.8932, the 0.893 the paper reports.

BibTeX

@article{delong1988comparing,
  author  = {DeLong, Elizabeth R. and DeLong, David M. and Clarke-Pearson, Daniel L.},
  title   = {Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach},
  journal = {Biometrics},
  volume  = {44}, number = {3}, pages = {837--845}, year = {1988},
  doi     = {10.2307/2531595}
}
@article{sun2014fast,
  author  = {Sun, Xu and Xu, Weichao},
  title   = {Fast implementation of {DeLong}'s algorithm for comparing the areas under correlated receiver operating characteristic curves},
  journal = {IEEE Signal Processing Letters},
  volume  = {21}, number = {11}, pages = {1389--1393}, year = {2014},
  doi     = {10.1109/LSP.2014.2337313}
}
@article{hanley1982meaning,
  author  = {Hanley, James A. and McNeil, Barbara J.},
  title   = {The meaning and use of the area under a receiver operating characteristic ({ROC}) curve},
  journal = {Radiology},
  volume  = {143}, number = {1}, pages = {29--36}, year = {1982},
  doi     = {10.1148/radiology.143.1.7063747}
}
@article{youden1950index,
  author  = {Youden, W. J.},
  title   = {Index for rating diagnostic tests},
  journal = {Cancer},
  volume  = {3}, number = {1}, pages = {32--35}, year = {1950},
  doi     = {10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3}
}
@article{wilson1927probable,
  author  = {Wilson, Edwin B.},
  title   = {Probable inference, the law of succession, and statistical inference},
  journal = {Journal of the American Statistical Association},
  volume  = {22}, number = {158}, pages = {209--212}, year = {1927},
  doi     = {10.1080/01621459.1927.10502953}
}
@article{simel1991likelihood,
  author  = {Simel, David L. and Samsa, Gregory P. and Matchar, David B.},
  title   = {Likelihood ratios with confidence: sample size estimation for diagnostic test studies},
  journal = {Journal of Clinical Epidemiology},
  volume  = {44}, number = {8}, pages = {763--770}, year = {1991},
  doi     = {10.1016/0895-4356(91)90128-V}
}
@article{mcclish1989analyzing,
  author  = {McClish, Donna Katzman},
  title   = {Analyzing a portion of the {ROC} curve},
  journal = {Medical Decision Making},
  volume  = {9}, number = {3}, pages = {190--195}, year = {1989},
  doi     = {10.1177/0272989X8900900307}
}
@article{vickers2006decision,
  author  = {Vickers, Andrew J. and Elkin, Elena B.},
  title   = {Decision curve analysis: a novel method for evaluating prediction models},
  journal = {Medical Decision Making},
  volume  = {26}, number = {6}, pages = {565--574}, year = {2006},
  doi     = {10.1177/0272989X06295361}
}

Limits

Questions

What does the AUC mean?

The area under the ROC curve is the chance that a randomly chosen case scores higher than a randomly chosen non-case, with ties counting half. 0.5 is a coin toss and 1 is perfect separation. It says nothing about calibration or about where to put the cut-off.

How do I compare two AUCs measured on the same patients?

Use DeLong's paired test. Two scores measured on the same people are correlated, so comparing their two confidence intervals, or using a test for independent samples, is too conservative. DeLong's method estimates the covariance of the two AUCs from the same data and tests the difference with it.

Why can two scores with overlapping intervals still differ?

Because the paired test uses the correlation between them. When two scores rank the same people in nearly the same order, the difference between their AUCs varies much less than either AUC, so a small difference can be significant even when the separate intervals overlap.

Which cut-off should I use?

It depends on what a missed case and a false alarm cost. Youden's J weighs sensitivity and specificity equally; a screening test usually wants a target sensitivity, a confirmatory test a target specificity. For a predicted probability, the decision curve shows which threshold gives the most net benefit for a given trade-off.

What is the partial AUC for?

Often only part of the curve matters, for example specificity above 90% because false positives are expensive. The partial AUC measures the area over that part only. McClish's standardisation puts it back on a 0.5 (useless) to 1 (perfect) scale, so two scores can be compared where it counts.

What is a decision curve?

For a model that predicts a probability, the net benefit at a threshold probability pt is true positives per person minus false positives per person weighted by pt/(1 - pt), the odds at which a person would accept treatment. The model is worth using where its curve is above both treating everyone and treating no one (Vickers and Elkin 2006).

Is my data uploaded anywhere?

No. The file is read and analysed by JavaScript in this browser tab. There are no accounts, cookies or analytics vendors.

Example data: the Breast Cancer Wisconsin (Diagnostic) data set of Wolberg, Mangasarian, Street and Street (1993), UCI Machine Learning Repository, doi:10.24432/C5DW2B, licensed CC BY 4.0, retrieved on 2026-11-19: 569 fine-needle aspirates of breast masses, 212 malignant and 357 benign. Kept: four of its thirty nuclear features, unchanged, and one added column, cv_probability, the out-of-fold probability of malignancy from a three-feature logistic regression fitted by 10-fold cross-validation (data/build_example.py), so no row's probability saw its own diagnosis.