Receiver Operating Characteristic (ROC) Curve

A receiver operating characteristic (ROC) curve is a plot that shows how well a binary classifier separates two classes across every possible decision threshold. It graphs the true positive rate on the y-axis against the false positive rate on the x-axis, with one point for each threshold applied to the model's scores, so a single picture covers every trade-off between catching positives and raising false alarms. Its area, the area under the curve (AUC), condenses that picture into one threshold-independent number. The name comes from World War II radar engineering, where it measured how well radar receiver operators told enemy aircraft apart from noise, and the method later spread to medicine, psychology, and machine learning (ML).

How an ROC Curve Is Built#

Every point on the curve comes from a confusion matrix computed at one threshold:

• True positive rate (TPR), also called recall or sensitivity: TP / (TP + FN), the share of actual positives the model flags.
• False positive rate (FPR): FP / (FP + TN), the share of actual negatives wrongly flagged. FPR equals 1 minus specificity.

Sweeping the threshold from high to low traces the curve from (0, 0), where nothing is flagged, to (1, 1), where everything is. A classifier that ranks every positive above every negative passes through the top-left corner, a model that guesses randomly follows the diagonal, and a curve below the diagonal means the scores are inverted. The AUC equals the probability that a randomly chosen positive receives a higher score than a randomly chosen negative, so 0.5 is chance and 1.0 is perfect separation. Building the curve requires continuous scores, such as predicted probabilities or confidence values, because hard 0/1 labels give only a single point. The following example trains a logistic regression classifier on an imbalanced dataset and computes its curve with scikit-learn's roc_curve:

import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score, roc_curve
from sklearn.model_selection import train_test_split

# Binary task where 10% of samples are positive
X, y = make_classification(n_samples=5000, weights=[0.9], class_sep=0.8, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, random_state=0)

# ROC analysis needs continuous scores, not hard 0/1 predictions
scores = LogisticRegression().fit(X_train, y_train).predict_proba(X_test)[:, 1]
fpr, tpr, thresholds = roc_curve(y_test, scores)
print(f"AUC: {roc_auc_score(y_test, scores):.3f}")

# Each threshold is one point on the curve: higher recall costs more false alarms
for target in (0.80, 0.95):
    i = np.argmax(tpr >= target)  # first point that reaches the target TPR
    print(f"Threshold {thresholds[i]:.2f}: TPR {tpr[i]:.2f}, FPR {fpr[i]:.2f}")

The model reaches an AUC of 0.963. At a threshold of 0.44 it finds 82% of positives with an FPR of 0.01, while reaching 95% recall requires lowering the threshold to 0.02, where the FPR jumps to 0.26. On this test split of 130 positives and 1,120 negatives, catching 18 more positives costs 279 more false alarms.

Choosing an Operating Point#

The curve shows what each threshold costs, and the application decides which cost is acceptable:

• Medical screening: A missed tumor is far costlier than a follow-up scan, so screening tools run at a high-TPR point and accept more false positives for radiologists to review.
• Fraud detection: Every blocked legitimate payment frustrates a customer, so banks pick a point that catches most fraud while holding the FPR to a level their review teams and customers can absorb.
• Equal costs: When both errors cost the same, Youden's J statistic, TPR minus FPR, picks the point farthest above the diagonal.

Ultralytics YOLO26 classification models return a probability for every class in results[0].probs.data, the continuous scores ROC analysis needs, so collecting one class's probability over a labeled validation set gives that class's one-vs-rest ROC curve and a threshold chosen for the deployment's costs.

ROC Curve vs. Related Tools#

A precision-recall curve plots precision against recall instead of TPR against FPR. Because TPR and FPR are each computed within one class, an ROC curve stays the same when positives become rarer, while precision falls. With rare positives, as in fraud screening or anomaly detection, a small FPR can still mean more false alarms than true detections, and a PLOS ONE study of imbalanced datasets found precision-recall plots more informative in that setting. Object detection is evaluated with precision-recall curves and mean average precision (mAP) for a related reason: the background regions that would count as true negatives are unbounded, so FPR has no fixed denominator.

• Accuracy and F1-score: Single numbers measured at one threshold, while the ROC curve and its AUC summarize all thresholds.
• Image classification with many classes: Each class gets a one-vs-rest ROC curve, and the per-class AUCs are averaged into a macro or prevalence-weighted score, which scikit-learn's roc_auc_score computes with multi_class="ovr".