Detection Head

A detection head is the final set of layers in an object detection network. It reads the feature maps produced by the earlier layers and outputs, at every candidate location, a bounding box and a score for each class. Its design decides how boxes are encoded, as offsets from anchor boxes or as distances from a point, how many candidate boxes the model produces, and whether a post-processing step such as non-maximum suppression (NMS) has to remove duplicates.

Where the Head Sits#

Modern detectors are built from three stages. A backbone, usually a convolutional network or a vision transformer, turns pixels into feature maps whose resolution falls and whose semantic content rises with depth. A neck, such as a feature pyramid network (FPN), merges deep and shallow maps so each output scale carries both fine detail and context. The head then runs small convolutional branches on every pyramid level, typically at strides 8, 16, and 32, so the fine maps find small objects and the coarse maps find large ones.

One-stage detectors have a single dense head that predicts at every grid cell. Two-stage detectors such as Faster R-CNN have two: a region proposal network that scores anchors at every location, then a region-of-interest head that classifies and refines the few hundred regions it proposes.

How a Head Predicts Boxes and Classes#

At each grid cell, a dense head predicts three kinds of values:

• Class scores: One per class, each passed through a sigmoid so a location can score each class independently.
• Box regression: Four numbers that place a box, either offsets that shift and resize an anchor box or the distances from the cell's center to the four box edges.
• Quality and task outputs: An objectness score (YOLOv1 through YOLOv5) or centerness score (FCOS) that ranks boxes, plus task-specific values such as mask coefficients for instance segmentation or keypoints for pose estimation.

The box encoding is what separates anchor-based detectors from anchor-free detectors. Each branch is a few convolutions ending in a 1 × 1 convolution with one output channel per predicted value. RetinaNet and FCOS already used separate classification and regression branches, and YOLOX brought this decoupled head to YOLO in 2021, reporting much faster training convergence than YOLOv3's single shared output layer. The sketch below builds those final 1 × 1 layers for the three pyramid levels of a 640 × 640 image and flattens their outputs into one list of candidates:

import torch
from torch import nn

# Neck outputs for a 640x640 image at strides 8, 16, and 32 (80x80, 40x40, and 20x20 grids)
channels = (64, 128, 256)
features = [torch.randn(1, c, 640 // s, 640 // s) for c, s in zip(channels, (8, 16, 32))]

# Decoupled head: a box branch (4 values) and a class branch (80 scores) for every pyramid level
box_branches = nn.ModuleList(nn.Conv2d(c, 4, 1) for c in channels)
cls_branches = nn.ModuleList(nn.Conv2d(c, 80, 1) for c in channels)

# Flatten each level's grid and concatenate, giving one candidate per grid cell
boxes = torch.cat([branch(f).flatten(2) for branch, f in zip(box_branches, features)], dim=2)
scores = torch.cat([branch(f).flatten(2) for branch, f in zip(cls_branches, features)], dim=2).sigmoid()
print(f"boxes {tuple(boxes.shape)}, class scores {tuple(scores.shape)}")

It prints boxes (1, 4, 8400), class scores (1, 80, 8400): the 6,400 + 1,600 + 400 grid cells each propose one box with 80 class scores.

From Raw Outputs to Detections#

Training decides how many predictions each object receives. One-to-many assignment marks several cells near each object as positives, which gives the head more training signal and leaves several overlapping boxes per object at inference, so NMS keeps the best one. One-to-one assignment, used by DETR and by the inference head of YOLOv10, trains exactly one prediction per object, so the outputs need no NMS.

The YOLO26 head predicts the four edge distances directly, dropping the 16-bin Distribution Focal Loss output per edge that YOLOv8 and YOLO11 used, and it trains a one-to-many and a one-to-one copy of its box and class branches. YOLO26 applies NMS to the one-to-many outputs by default, and passing nms=False switches to the one-to-one branches, which return at most 300 boxes per image with no NMS step.

Head Design in Practice#

• Fine-tuning on new classes: The class branch ends in one output channel per class, so training on a dataset with a different number of classes needs a new final classification layer, while the pretrained backbone and neck weights carry over.
• Deployment: Every operation in the head runs on the target device, so simple box decoding and NMS-free outputs make export easier and cut postprocessing time.
• Small objects: Adding a stride-4 pyramid level gives the head a finer grid for tiny objects at extra compute cost: for a 640 × 640 input it adds 160 × 160 = 25,600 cells, four times as many as the stride-8 level.