Receptive Field

The receptive field of a unit in a convolutional neural network (CNN) is the region of the input image that can influence that unit's value. A unit in a network's first 3 × 3 convolution layer sees a 3 × 3 patch of pixels, and units in deeper layers see progressively larger regions because each layer combines neighboring outputs of the one before. The receptive field sets how much context a feature can use: a unit that recognizes a whole bus must see the whole bus. The term comes from neuroscience, where it names the area of the visual field that drives a single neuron.

How the Receptive Field Grows#

Each layer adds (k − 1) × j pixels to the receptive field, where k is its kernel size and j, the jump, is the distance in input pixels between neighboring units of its input, equal to the product of all earlier strides. Three consequences follow:

• Stacking small kernels: Three 3 × 3 layers at stride 1 see a 7 × 7 patch, the same as one 7 × 7 layer, with 27C² weights instead of 49C² for C channels, the argument the VGG networks used for building deep stacks of 3 × 3 convolutions.
• Striding and pooling: Every stride-2 layer or pooling step doubles the jump, so later layers grow the receptive field twice as fast. This is how CNNs reach receptive fields of hundreds of pixels while shrinking their feature maps.
• Dilation: A dilated (atrous) 3 × 3 convolution with dilation d spaces its taps d pixels apart and covers 2d + 1 pixels, enlarging the receptive field without downsampling, which suits dense per-pixel tasks such as semantic segmentation.

Measuring the receptive field directly confirms these rules: backpropagate from one output value and see which input pixels receive a nonzero gradient.

import torch
from torch import nn

# Three stacked 3x3 convolutions in three configurations: (stride, dilation)
for name, stride, dilation in (("plain", 1, 1), ("strided", 2, 1), ("dilated", 1, 2)):
    convs = [nn.Conv2d(1, 1, 3, stride, padding=dilation, dilation=dilation, bias=False) for _ in range(3)]
    for conv in convs:
        nn.init.ones_(conv.weight)  # positive weights, so no input pixel's gradient cancels out
    layers = nn.Sequential(*convs)

    # Backpropagate from the central output value; input pixels with nonzero gradient form its receptive field
    image = torch.zeros(1, 1, 64, 64, requires_grad=True)
    output = layers(image)
    output[0, 0, output.shape[2] // 2, output.shape[3] // 2].backward()
    rows = image.grad[0, 0].sum(dim=1).nonzero()
    size = rows.max().item() - rows.min().item() + 1
    print(f"{name}: {size}x{size} pixels")

It prints 7 × 7 for the plain stack, 15 × 15 for the strided stack, where the jump grows from 1 to 2 to 4, and 13 × 13 for the dilated stack, where each layer adds 4 pixels.

Theoretical vs. Effective Receptive Field#

The formula gives the theoretical receptive field, every pixel that can affect a unit. The effective receptive field measures how much each pixel actually does: its influence has a roughly Gaussian shape concentrated at the center, and it occupies only a fraction of the theoretical area. Larger kernels widen it; RepLKNet used depthwise kernels up to 31 × 31 and reported much larger effective receptive fields than small-kernel CNNs. In vision transformers, self-attention lets every image patch draw on every other patch from the first layer, so the theoretical receptive field covers the whole image at every depth.

Receptive Field vs. Related Terms#

• Kernel size: The window one convolution reads from its own input, such as 3 × 3. The receptive field is the accumulated area in the original image, and three stacked 3 × 3 kernels already give 7 × 7.
• Feature map: The output grid of a layer. The receptive field describes the input region behind a single position on that grid.
• Context window: The number of tokens a language model reads at once, the sequence counterpart of a receptive field on an image grid.

Receptive Fields in Object Detection#

Detectors predict small objects from fine feature maps and large objects from coarse, deep maps with large receptive fields, and a feature pyramid network (FPN) mixes deep context back into the fine maps. A receptive field smaller than an object sees only part of it, and tiny objects still need a fine level, because on a stride-32 map a 20-pixel object spans less than one grid cell. YOLO26 predicts from maps at strides 8, 16, and 32, and at the end of its backbone it applies an SPPF block, which max-pools the stride-32 map with windows equivalent to 5 × 5, 9 × 9, and 13 × 13 cells, followed by a self-attention block whose reach spans the whole stride-32 map.