License YOLO
Back to Ultralytics Glossary

Identity Mapping

Identity mapping passes a block's input unchanged through a shortcut and adds it to the output, so gradients reach the early layers of deep networks.

In deep learning, an identity mapping is a connection that carries a block's input forward unchanged and adds it to the block's output, so the block computes y = x + F(x) instead of y = F(x). The name comes from the identity function, f(x) = x. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun built residual networks (ResNet) on these identity shortcuts in 2015 and won that year's ImageNet classification challenge with a 152-layer network, and identity shortcuts now run through nearly every deep architecture, from convolutional neural networks (CNNs) to the residual stream of every transformer.

Why Identity Mappings Make Depth Trainable#

Before ResNet, adding layers often made networks worse. He and colleagues found that a 56-layer plain network had higher training error on CIFAR-10 than a 20-layer one, so the extra depth hurt optimization itself, a problem they called degradation. A deeper network can always match a shallower one by setting its extra layers to the identity, yet stacks of nonlinear layers struggle to learn the identity. A residual block only has to learn the difference F(x) = H(x) - x from its input, and driving F toward zero recovers the identity.

The shortcut also changes backpropagation. Because y = x + F(x), the derivative of y with respect to x is 1 plus the derivative of F, and that constant 1 passes the gradient straight back through every block. In a plain stack, the gradient is a product of many per-layer factors, and when those factors are below 1 the result shrinks exponentially, the vanishing gradient problem. Running the same 50 layers with and without shortcuts in PyTorch shows the difference:

import torch
from torch import nn

torch.manual_seed(0)
# 50 identical-shape layers, reused for a plain and a residual stack
layers = [nn.Sequential(nn.Linear(64, 64), nn.Tanh()) for _ in range(50)]
x = torch.randn(32, 64, requires_grad=True)

for residual in (False, True):
    h = x
    for layer in layers:
        h = h + layer(h) if residual else layer(h)  # identity shortcut adds the input back
    loss = h.square().mean()
    (grad,) = torch.autograd.grad(loss, x)
    print(f"residual={residual}: gradient norm at the input {grad.norm():.1e}")

It prints a gradient norm of 1.1e-15 at the input for the plain stack and 2.8e+00 for the residual stack built from the same weights. Without shortcuts, the first layers of this network would receive almost no training signal.

Keeping the Shortcut Clean#

In their 2016 follow-up, Identity Mappings in Deep Residual Networks, the same authors tested shortcuts that scale, gate, convolve, or apply dropout to x, and every variant trained worse than a pure identity. They also moved batch normalization and the activation function inside the residual branch, before the weights, so nothing touches the signal between blocks. This pre-activation design trained a 1,001-layer ResNet on CIFAR-10 to 4.62% error and a 200-layer ResNet on ImageNet. Modern transformers follow the same rule with pre-norm blocks, which apply normalization inside each attention and feed-forward branch and leave the residual stream itself untouched.

An identity shortcut needs the input and output to have the same shape. Where a block changes the number of channels or downsamples, ResNet substitutes a projection shortcut, usually a 1 x 1 convolution with matching stride:

Ultralytics YOLO26 uses identity shortcuts throughout its backbone and neck: every C3k2 block is built from bottlenecks that return x plus the output of two 3 x 3 convolutions whenever their input and output channels match, and its SPPF block adds its input back to its pooled output the same way.

Identity Initialization#

A block can also start as an identity. Setting the scale of the last batch normalization layer in each residual branch to zero makes F(x) = 0 at initialization, so a deep network begins training as if it were shallow; Priya Goyal and colleagues used this to train ResNet-50 on ImageNet in one hour with large batches, and torchvision exposes it as zero_init_residual=True. ReZero, proposed in 2020, multiplies each residual branch by a learnable scalar that starts at zero.

PyTorch's nn.Identity module is a literal identity map with no parameters. Replacing a classifier head with it, as in model.fc = nn.Identity(), turns an image classification network into a feature extraction model that returns its penultimate embeddings.

Skip connection is the broader term for any path that bypasses layers. A residual connection is the additive kind, and an identity mapping is a residual connection whose path applies no transformation. U-Net and DenseNet use skip connections that concatenate earlier feature maps instead of adding them. Identity preservation is an unrelated term from generative AI and tracking, meaning that a generated or tracked person keeps the same recognizable appearance across images or video frames.

Explore solutions

Computer vision in sports

Computer vision in sports

Discover how Ultralytics YOLO powers sports vision with player and ball tracking, performance analytics, coaching insights, and athlete rehabilitation.
Learn more
Computer vision for aerial imagery

Computer vision for aerial imagery

Turn drone and aerial imagery into real-time insights for agriculture, aquaculture, conservation, and geospatial analysis with Ultralytics YOLO.
Learn more
Computer vision in aerospace

Computer vision in aerospace

Apply computer vision in aerospace with Ultralytics YOLO for aircraft inspection, runway FOD detection, airport ground operations, and composite defect detection.
Learn more
Computer vision in security

Computer vision in security

Protect every site with Ultralytics YOLO models. Vision AI powers perimeter monitoring, threat detection, license-plate recognition, and workplace safety.
Learn more
Computer vision in robotics

Computer vision in robotics

Power smarter machines with Ultralytics YOLO models. Vision AI in robotics drives autonomous navigation, perception, object tracking, and real-time control.
Learn more
Computer vision in logistics

Computer vision in logistics

Streamline logistics with Ultralytics YOLO models. Vision AI enables package inspection, sorting, vehicle tracking, and real-time warehouse safety monitoring.
Learn more
Computer vision in retail

Computer vision in retail

Reimagine retail with Ultralytics YOLO models. Vision AI powers inventory tracking, shelf monitoring, queue management, and smarter customer insights.
Learn more
Computer vision in healthcare

Computer vision in healthcare

Build healthcare solutions with Ultralytics YOLO models. Vision AI in healthcare powers faster medical imaging, smarter diagnostics, and patient monitoring.
Learn more
Computer vision in manufacturing

Computer vision in manufacturing

Optimize manufacturing with Ultralytics YOLO models. Vision AI drives quality control, defect detection, PPE compliance, and assembly line automation.
Learn more
Computer vision in automotive

Computer vision in automotive

Apply computer vision in automotive with Ultralytics YOLO models. Vision AI elevates road safety, driver assistance, and vehicle automation for smarter roads.
Learn more
Computer vision in agriculture

Computer vision in agriculture

Bring vision AI to smart agriculture with Ultralytics YOLO models. Power crop monitoring, livestock tracking, and precision farming for higher, smarter yields.
Learn more

Let's build the future of AI together!

Begin your journey with the future of machine learning