Intersection over Union (IoU)

Intersection over union (IoU) measures how well two regions overlap: the area they share divided by the total area they cover together. It ranges from 0 for regions that do not touch to 1 for identical regions, and it is the same quantity known in statistics as the Jaccard index. In object detection, IoU compares a predicted bounding box with the labeled ground-truth box, and it decides whether a prediction counts as correct, which duplicate predictions get removed, and how box predictions are trained.

How IoU Is Calculated#

IoU = area of intersection / area of union, where the union is the area of box A plus the area of box B minus their intersection, so the shared region is counted once. For two axis-aligned boxes in corner format, the intersection is itself a rectangle bounded by the inner edges of the two boxes, and its width or height is clamped to zero when the boxes do not overlap.

# Boxes as [x1, y1, x2, y2] in pixels
label = [100, 100, 200, 200]
pred = [110, 110, 210, 210]

# Intersection rectangle, clamped to zero when the boxes do not overlap
w = max(0, min(label[2], pred[2]) - max(label[0], pred[0]))
h = max(0, min(label[3], pred[3]) - max(label[1], pred[1]))
intersection = w * h

# Union counts the overlap only once
area_label = (label[2] - label[0]) * (label[3] - label[1])
area_pred = (pred[2] - pred[0]) * (pred[3] - pred[1])
union = area_label + area_pred - intersection

print(f"IoU = {intersection} / {union} = {intersection / union:.3f}")

The script prints IoU = 8100 / 11900 = 0.681. Shifting a 100-pixel box by 10 pixels in each direction shrinks the overlap to 90 × 90 pixels and grows the union to 11,900, so a 10% shift costs almost a third of the score. Because both areas scale together, IoU is scale-invariant: the same relative misalignment gives the same score on a 20-pixel bird and a 2,000-pixel truck. A fixed pixel error therefore hurts small objects most, and a one-pixel sideways shift lowers IoU to 0.905 on a 20-pixel box and to 0.990 on a 200-pixel box.

IoU Thresholds in Evaluation#

An evaluation threshold turns IoU into a yes-or-no decision. A prediction is a true positive when its class matches and its IoU with a not-yet-matched ground-truth box reaches the threshold, and a false positive otherwise. The PASCAL VOC benchmark used 0.5, a looser bar than it sounds: two equal boxes offset sideways by a third of their width have an IoU of exactly 0.5. The COCO benchmark averages its results over ten thresholds from 0.50 to 0.95, which is how mean average precision (mAP) rewards tight localization.

IoU in Training and Post-Processing#

• Box regression losses: Detectors train box coordinates with 1 − IoU or a variant of it, because a loss on raw coordinates treats a 10-pixel error the same on small and large boxes. Plain IoU gives no gradient when the boxes do not overlap, so Generalized IoU (GIoU) adds a penalty based on the smallest box enclosing both, and Distance-IoU and Complete IoU (DIoU, CIoU) add terms for center distance and aspect ratio. YOLO26 trains its box head with a CIoU loss and also uses CIoU to decide which predictions are assigned to each labeled object during training.
• Non-maximum suppression (NMS): At inference, a box whose IoU with a higher-scoring box of the same class exceeds a threshold is treated as a duplicate and removed.
• Object tracking: Trackers such as ByteTrack use the IoU between existing tracks and new detections as a matching cost from frame to frame.

IoU for Other Shapes#

The same ratio applies to any pair of regions. Segmentation computes mask IoU over sets of pixels: instance segmentation uses it to match predicted masks to labels, and semantic segmentation benchmarks report its mean over classes as mIoU. Rotated boxes need a polygon intersection or a Gaussian approximation such as ProbIoU, and 3D detection computes IoU between cuboids, either in full 3D or as the overlap of their footprints in a bird's-eye view.

IoU vs. Related Metrics#

• Dice coefficient: twice the intersection divided by the sum of the two areas. It ranks predictions in the same order as IoU because Dice = 2 × IoU / (1 + IoU), and it is common in medical image segmentation.
• Accuracy: the share of correct class predictions, which says nothing about where a box sits.
• mAP: a detection-level summary built on top of IoU thresholds, combining localization with precision and recall across classes.

The Dice relation also converts thresholds: an IoU of 0.5 equals a Dice score of 0.667, and an IoU of 0.9 equals a Dice score of 0.947.