Backbone

A backbone is the part of a neural network that turns raw input into feature maps for the rest of the model. In computer vision, the backbone is usually a convolutional neural network (CNN) or a vision transformer (ViT) pretrained on a large dataset such as ImageNet. The layers after it, typically a neck and a task-specific head, convert its features into class labels, bounding boxes, masks, or keypoints. Backbone, encoder, and feature extractor are often used as synonyms; encoder is the usual name in encoder-decoder models such as U-Net.

How a Backbone Works#

A vision backbone processes an image in stages. Each stage applies a stack of convolution or attention blocks, then halves the spatial resolution and usually increases the number of channels, so a 640×640 input becomes an 80×80 map at stride 8, a 40×40 map at stride 16, and a 20×20 map at stride 32. Early stages respond to edges, colors, and textures. Later stages respond to object parts and whole objects, because each of their output positions covers a larger receptive field of the input image. Detection and segmentation models tap the last three stages and pass them to a neck such as a feature pyramid network (FPN), which lets the model find small objects in the high-resolution map and large objects in the low-resolution one.

Backbone vs. Neck vs. Head#

Most modern object detection architectures split into three parts, and the boundaries matter when you swap components or freeze layers.

• Backbone: Performs the feature extraction, producing feature maps at several resolutions. It holds general visual knowledge that transfers across tasks and datasets.
• Neck: Fuses the backbone's scales so each output level carries both fine spatial detail and high-level meaning. FPN adds a top-down path, and PANet adds a second, bottom-up path after it. Classification models usually have no neck and pool the final backbone stage into a single vector.
• Head: Turns each fused feature map into task outputs. A detection head predicts boxes and class scores, and segmentation and pose heads predict masks and keypoints from the same kind of features.

Common Backbone Families#

• ResNet: Residual connections let the signal skip blocks, making networks up to 152 layers deep trainable, and a ResNet ensemble won the ILSVRC 2015 classification task. ResNet-50 is still a standard reference backbone for comparing detectors.
• CSPNet: Cross Stage Partial networks send only part of each stage's channels through the stage's blocks and merge the rest back at the end, which cut computation by 20% at equal or better ImageNet accuracy. CSPDarknet53, the YOLOv4 backbone, is built from these stages.
• ConvNeXt: A ResNet modernized with design choices borrowed from transformers, such as large kernels and fewer normalization layers, reaching 87.8% ImageNet top-1 accuracy and outperforming Swin Transformer on COCO detection.
• ViT and Swin Transformer: A plain ViT keeps a single resolution from start to finish, so detection models often use Swin instead, which computes attention inside shifted local windows and produces a CNN-like hierarchy of stages.
• Self-supervised foundation backbones: DINOv3 trains a 7-billion-parameter ViT on 1.7 billion unlabeled images, and its features reach strong results on dense tasks such as segmentation and depth estimation with the backbone kept frozen.

Pretrained Backbones and Transfer Learning#

Training a backbone from scratch needs a large dataset and a long schedule, so most projects start from pretrained weights. This kind of transfer learning works because the edges, textures, and shapes learned on one dataset are useful on most others. With a small dataset close to the pretraining domain, freezing the backbone keeps those general features intact, reduces overfitting, and lowers training memory, since frozen layers store no gradients or optimizer state. With a large dataset or a distant domain, such as X-ray or thermal imagery, fine-tuning the backbone too lets its features adapt. In YOLO26, the backbone spans layers 0–10 (strided convolutions, C3k2 blocks, an SPPF pooling block, and a C2PSA attention block), and the freeze=11 training argument freezes all of them so training updates only the neck and the detection head, a setup the fine-tuning guide covers in detail.

from ultralytics import YOLO

# Load a COCO-pretrained YOLO26 detection model
model = YOLO("yolo26n.pt")

# Fine-tune on a small dataset with the whole backbone (layers 0-10) frozen
results = model.train(data="coco8.yaml", epochs=3, imgsz=640, freeze=11)

The trainer logs Freezing layer 'model.0.conv.weight' before the first epoch, plus a matching line for every other parameter in layers 0–10, the parameters that stay fixed while the neck and head learn the new data.