CutMix

CutMix is a data augmentation technique that cuts a rectangular patch from one training image, pastes it over the same region of another, and mixes the two labels in proportion to the area each image occupies. Sangdoo Yun and colleagues at NAVER Clova AI Research introduced it in a 2019 ICCV paper as a combination of two earlier ideas: Cutout's removal of a random region and MixUp's mixing of labels. On ImageNet, CutMix lowered the top-1 error of a ResNet-50 from 23.68% to 21.40%, ahead of MixUp at 22.58% and Cutout at 22.93%, and models pretrained with it also transferred better to object detection and image captioning.

How CutMix Works#

For each pair of images A and B, CutMix samples a mixing ratio λ from a Beta distribution, which with the default parameter α = 1 is uniform between 0 and 1. It then draws a box whose width and height are the image's width and height times √(1 − λ), centers it at a random point, clips it to the image borders, and copies that region of B into A. After clipping, λ is recomputed as the fraction of A that remains, and the label becomes λ parts A and (1 − λ) parts B, so a 30% patch of a cat on a dog image yields a target of 0.7 dog and 0.3 cat. Training minimizes the loss function against this soft label, the same way as with label smoothing.

PyTorch's torchvision applies CutMix to a whole batch, pairing each image with its neighbor in the batch:

import torch
from torchvision.transforms import v2

# A batch of 4 random 224x224 images labeled with classes 0-3 of 10
torch.manual_seed(0)
images = torch.rand(4, 3, 224, 224)
labels = torch.tensor([0, 1, 2, 3])

# Paste a random box from each image's neighbor in the batch and mix the labels by area
cutmix = v2.CutMix(num_classes=10)
mixed_images, mixed_labels = cutmix(images, labels)

# Image 0 received its patch from image 3
pasted = (mixed_images[0] != images[0]).any(dim=0).float().mean()
print(f"pasted area: {pasted:.3f}")
print(f"label weights for classes 0-3: {mixed_labels[0, :4].numpy().round(3)}")

It prints a pasted area of 0.187 and label weights of 0.813 for class 0 and 0.187 for class 3: the share of the target assigned to each class equals the share of pixels it contributes.

Why CutMix Helps#

An image classification model trained on whole images can succeed by recognizing one small, highly discriminative part, such as a dog's face. When CutMix covers that part with another image, the model has to recognize the dog from its body, legs, or fur, so it learns to use more of each object, which improves weakly supervised localization and robustness to occlusion. Every pixel of the mixed image still comes from a real photo, so the model never trains on blank regions or ghostly blends. Like other strong augmentations, CutMix acts as regularization that reduces overfitting, and modern training recipes for vision transformers (ViT), including the one behind Meta's Data-efficient Image Transformers (DeiT), apply CutMix and MixUp together.

CutMix vs. Related Augmentations#

• MixUp: Blends two whole images pixel by pixel with weight λ and mixes their labels the same way, which produces semi-transparent overlays that look unlike any real scene.
• Cutout: Masks a random square with zeros and keeps the original label, so the masked pixels carry no information for training.
• Mosaic: Tiles four resized images into one, a standard augmentation for training object detectors that keeps the box label of every object still in the frame.
• Copy-paste for instance segmentation: Pastes segmented object instances, rather than rectangles, from one image into another.

CutMix in Object Detection#

Detection keeps every label hard: instead of a mixed class vector, the boxes of both images are kept where their objects remain visible. The Ultralytics YOLO cutmix augmentation, off by default, takes a probability between 0 and 1. Its implementation chooses a cut region that overlaps none of the target image's existing boxes, copies the same region from a second image, and keeps each pasted bounding box that retains at least 10% of its area inside the region, or 1% for segmentation labels. CutMix switches off with mosaic for the final 10 epochs of training by default, set by close_mosaic=10.