Data Augmentation

Data augmentation increases the variety of a training dataset by applying random transformations, such as flips, crops, rescaling, color shifts, and image mixing, to the examples it already contains. Each transformed copy keeps its original label, so the model learns that an object stays the same object under changes in viewpoint, lighting, and position. Collecting and labeling new images is slow and expensive, which makes augmentation the cheapest defense against overfitting, and nearly every modern computer vision model trains with it.

How Data Augmentation Works#

Most training pipelines augment online: the data loader transforms each image as it builds a batch, drawing new random parameters every time, so each epoch sees a different version of every image. Offline augmentation instead writes a fixed number of transformed copies to disk before training, which multiplies storage and gives the model far fewer distinct variants.

Two rules keep augmentation from corrupting the data. First, labels must move with the pixels: in object detection, instance segmentation, and pose estimation, a geometric transform has to update every bounding box, mask, and keypoint, and a horizontal flip also swaps left and right keypoints. Second, the transform must be one the real world could produce: vertical flips suit aerial and microscope images, while for street scenes they create upside-down cars that never appear at inference time. Torchvision's transforms handle the first rule automatically when boxes are passed alongside the image:

from urllib.request import urlopen

from PIL import Image
from torchvision import tv_tensors
from torchvision.transforms import v2

image = Image.open(urlopen("https://ultralytics.com/images/bus.jpg"))  # 810 x 1080 pixels
boxes = tv_tensors.BoundingBoxes([[50, 400, 250, 900]], format="XYXY", canvas_size=(1080, 810))

# Flip every time (p=1.0) and shift colors; the box must move with the pixels
augment = v2.Compose([v2.RandomHorizontalFlip(p=1.0), v2.ColorJitter(brightness=0.4, saturation=0.7)])
new_image, new_boxes = augment(image, boxes)

print(f"Original box: {boxes.tolist()}")
print(f"Flipped box:  {new_boxes.tolist()}")

The box around the man on the left of the image moves from [50, 400, 250, 900] to [560, 400, 760, 900], mirrored across the 810-pixel width, while the color jitter changes the pixels without touching the box.

Common Techniques#

• Geometric transforms: Flips, rotations, scaling, translation, cropping, shearing, and perspective warps simulate different camera positions and object sizes.
• Photometric transforms: Changes to brightness, contrast, saturation, and hue, plus blur and noise, simulate different lighting, sensors, and image quality.
• Occlusion: Random erasing and cutout blank out rectangles so the model cannot rely on any single part of an object.
• Image mixing: MixUp blends two images and their labels, CutMix pastes a patch from one image into another, mosaic tiles four images into one training sample, and copy-paste moves segmented objects onto new backgrounds.
• Automated policies: RandAugment applies a random sequence of transforms controlled by just two settings, the number of transforms and their strength, which removes the costly search that earlier learned policies needed.

Augmentation Pipelines for Object Detection#

Detection pipelines chain several transforms, each applied with its own probability, so every training image receives a different random combination.

Ultralytics YOLO training runs this pipeline by default, with mosaic on every image (mosaic=1.0), a 50% chance of a horizontal flip (fliplr=0.5), up to ±50% scaling, and HSV color shifts, and it turns mosaic off for the final 10 epochs (close_mosaic=10) so the model finishes on complete, natural-looking images.

from ultralytics import YOLO

# Load a pretrained YOLO26n model
model = YOLO("yolo26n.pt")

# fliplr: horizontal flip probability; scale: zoom range (+/- 70%); degrees: rotation range (+/- 5)
results = model.train(data="coco8.yaml", epochs=10, fliplr=0.5, scale=0.7, degrees=5.0)

During this run, the YOLO26 trainer saves its first three augmented batches as train_batch0.jpg, train_batch1.jpg, and train_batch2.jpg, with boxes drawn, so the effect of each setting can be checked before committing to a long run.

Data Augmentation vs. Related Terms#

• Synthetic data: Generated from scratch with rendering engines, simulators, or diffusion models. Augmentation modifies real images that already exist.
• Data preprocessing: Deterministic steps such as resizing and normalization applied identically to every image during training and inference. Augmentation is random and runs only during training.
• Test-time augmentation (TTA): Runs inference on several augmented versions of the same input, such as flipped and rescaled copies, and merges the predictions, trading extra compute for a small accuracy gain.