Area Under the Curve (AUC)

Area under the curve (AUC) is a metric that summarizes how well a binary classifier separates two classes across every possible decision threshold. In machine learning, AUC almost always means the area under the receiver operating characteristic (ROC) curve, also written ROC AUC or AUROC. Its value equals the probability that the model scores a randomly chosen positive example higher than a randomly chosen negative one. An AUC of 1.0 means every positive outranks every negative, 0.5 means the ranking is no better than chance, and values below 0.5 mean the ranking is inverted. Because it depends only on the order of the scores, AUC evaluates a model before anyone picks a threshold, which makes it a standard metric for medical diagnostic models, credit scoring, and fraud detection.

How AUC Is Calculated#

The ROC curve plots the true positive rate, which is recall, against the false positive rate, the share of negatives wrongly flagged, as the decision threshold sweeps from the highest score to the lowest. A threshold above every score flags nothing and sits at (0, 0); a threshold below every score flags everything and sits at (1, 1). AUC is the area under the line connecting all the points between, usually computed with the trapezoidal rule.

The same number comes from counting pairs. Comparing every positive with every negative and scoring each pair 1 when the positive ranks higher, 0.5 for a tie, and 0 otherwise gives exactly the ROC AUC, a quantity statisticians know as the normalized Mann-Whitney U statistic and medical researchers as the concordance or c-statistic. Credit risk teams often report the Gini coefficient instead, which equals 2 × AUC − 1.

The pairwise view explains AUC's main properties. Any transformation that keeps the order of the scores, such as squaring probabilities or applying temperature scaling to a binary classifier's logits, leaves AUC unchanged, so a model can have an excellent AUC and badly calibrated probabilities. Calibration needs a separate metric such as the Brier score.

ROC AUC vs. PR AUC on Imbalanced Data#

ROC AUC does not depend on the class ratio, because the true positive rate is computed only over positives and the false positive rate only over negatives. That property becomes a weakness when positives are rare. With 1 positive in 100, a false positive rate of 5% produces about five false alarms for every real positive, yet barely dents the ROC curve. The area under the precision-recall curve (PR AUC), usually summarized as average precision (AP), plots precision against recall and exposes those false alarms directly, which makes it more informative than the ROC curve on imbalanced data. The scikit-learn script below scores one model both ways:

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import average_precision_score, roc_auc_score
from sklearn.model_selection import train_test_split

# Imbalanced binary problem: 1 positive per 100 samples
X, y = make_classification(n_samples=20_000, weights=[0.99], flip_y=0, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, stratify=y, random_state=0)

# Score the test set with a simple classifier
model = LogisticRegression(max_iter=1000).fit(X_train, y_train)
scores = model.predict_proba(X_test)[:, 1]

# The same scores summarized by the areas under two different curves
print(f"Positives in test set: {y_test.sum()} of {len(y_test)}")
print(f"ROC AUC: {roc_auc_score(y_test, scores):.3f}")
print(f"PR AUC (average precision): {average_precision_score(y_test, scores):.3f}")

With 100 positives among 10,000 test samples, the model reaches a ROC AUC of 0.911 but an average precision of only 0.462. The two numbers also have different baselines: a random ranking scores 0.5 ROC AUC regardless of class balance, while its average precision equals the positive rate, 0.01 here.

Multi-Class AUC and Object Detection#

ROC AUC is defined for two classes. For more, scikit-learn's roc_auc_score averages one-vs-rest AUCs, where each class is scored against all others, or one-vs-one AUCs over every pair of classes, with macro or prevalence-weighted averaging.

Object detection cannot use ROC curves, because the negatives are all the box locations that contain no object, an effectively unbounded set with no meaningful false positive rate. Detection metrics use areas under precision-recall curves instead. YOLO26 validation in Ultralytics computes each class's average precision as the area under its precision-recall envelope, sampled at 101 recall points as in the COCO evaluation, and averages those areas into mean average precision (mAP) at IoU thresholds from 0.50 to 0.95.

AUC vs. Related Terms#

• Accuracy: Measures correct predictions at one threshold and can look excellent on imbalanced data while missing every positive. AUC measures the ranking across all thresholds.
• F1-score: Combines precision and recall at a single threshold, so it scores one operating point, while AUC summarizes all of them.
• Confusion matrix: Counts true and false positives and negatives at one threshold. Each ROC point summarizes one confusion matrix as a pair of rates.
• Average precision: The area under the precision-recall curve, the usual summary when positives are rare and the main ingredient of mAP.
• Partial AUC: The area over a restricted false positive range, such as 0 to 0.1, used when only low false-alarm rates are acceptable, as in large screening programs where every false alarm triggers a follow-up exam.