How do you use ROC curves to evaluate model performance?
Instruction: Explain what ROC curves are and how they can be used to assess the performance of classification models.
Updated
Official answer
A binary ROC curve plots the true-positive rate TP/(TP+FN) against the false-positive rate FP/(FP+TN) as the score threshold changes. Compute it from held-out scores and observed labels, with the positive class stated explicitly.
ROC AUC summarizes ranking across thresholds. A value of 1 means perfect separation on the evaluated sample; it does not prove perfect calibration or future performance. An AUC of 0.5 can occur for uninformative ranking, but the scalar alone does not describe every shape of ROC curve.
Choose an operating threshold using validation data and the real costs or capacity. For example, with 100 positives and 9,900 negatives, 80% sensitivity and a 1% false-positive rate produce 80 true positives and 99 false positives. Precision is only 80/179 ≈ 44.7%, despite the small false-positive rate.
Report the confusion matrix and precision-recall behavior at relevant thresholds, plus uncertainty. ROC rates condition on the class, but deployment prevalence changes the resulting counts and predictive values. Maximizing total AUC alone may miss the region that matters operationally.