Image Registration

Image registration is the process of aligning two or more images so that the same location in a scene falls at corresponding coordinates. Imagine placing a satellite image taken in June over one taken in January: if the camera angle or position changed, a road may appear to shift even though it has not moved. Registration corrects that geometric mismatch before a computer vision system compares the images or combines their information.

How Image Registration Works#

A registration workflow designates one image as the fixed image, which defines the target coordinate system, and another as the moving image, which must be aligned to it. The system estimates a spatial transformation, then resamples the moving image—calculating pixel values at its new positions. The result can be inspected as an overlay: stable structures should line up, while genuine changes remain visible. The ImageJ registration overview illustrates this relationship between corresponding points, transformations, and aligned images.

There are two common ways to estimate the transformation. Feature-based registration identifies recognizable locations, such as corners, in both images and matches points that represent the same place. It then fits a transformation to those correspondences, rejecting mismatches when possible; OpenCV’s feature matching and homography guide shows how this works. Intensity-based registration instead adjusts the transformation to improve a measure of similarity between the images’ pixel values. That measure and the search procedure, called an optimizer, are central to the intensity-based registration workflow.

Choosing a Transformation#

The appropriate transformation depends on why the images differ. A translation handles a simple horizontal or vertical shift. A rigid transformation also allows rotation while preserving distances. Affine registration additionally handles scaling and skew, but it applies one transformation across the whole image. A projective transformation, or homography, can account for perspective differences when the matched scene is approximately planar. The geometric transformation types explained by MathWorks provide useful visual distinctions.

Some scenes need deformable registration, which allows different regions to move by different amounts—for example, when anatomy changes shape between scans. This flexibility can improve local alignment, but an implausible warp can hide a real difference. Medical-image tools such as SimpleITK’s registration framework therefore make the transformation, similarity measure, optimizer, and interpolation method explicit.

Registration and Related Vision Tasks#

Registration is about a shared coordinate system, not simply visual resemblance. Image matching finds corresponding visual content and may supply the point pairs needed for registration; registration goes further by estimating and applying the spatial alignment. Keypoints can serve as those correspondences, although points on moving people or vehicles are usually poor anchors for aligning a stationary background.

Likewise, optical flow estimates apparent motion across frames. Its motion estimates can help register frames, as demonstrated in scikit-image’s optical-flow registration example, but flow is not itself a guarantee of accurate alignment. Object tracking has a different goal: maintaining an object’s identity over time. A tracker may use background motion estimates to avoid confusing camera movement with object movement.

Where It Matters in AI Applications#

In satellite change detection, an analyst may compare images of farmland captured months apart to identify newly cleared areas. Registration places fields, roads, and boundaries in the same pixel locations before a change-detection model compares them. Otherwise, a shifted field boundary can look like land-cover change. Differences in season, illumination, or sensor type still require care after alignment. This is a common concern in satellite image analysis.

In medical image analysis, a clinician may compare scans from separate visits to assess whether a lesion has changed. Aligning the scans makes the same anatomical region easier to compare or pass to a segmentation model. Because anatomy can deform and imaging methods can produce different brightness patterns, good registration must preserve clinically meaningful differences rather than force every region to look alike.

Practical Registration and Validation#

Start with the simplest transformation that plausibly explains the mismatch. Confirm that the images share enough visible content, account for differing resolutions, and inspect alignment at several locations—not just where the match is strongest. Save the estimated transformation when later predictions or annotations must be mapped back to the original image. OpenCV’s geometric transformation guide explains how image warping places transformed pixels.

The following self-contained example uses the scikit-image phase-correlation workflow to estimate a translation. Install its dependencies with pip install scikit-image scipy.

import numpy as np
from scipy.ndimage import shift
from skimage import data
from skimage.io import imsave
from skimage.registration import phase_cross_correlation

# Create a reference and a shifted copy of the same image.
fixed = data.camera()
moving = np.roll(fixed, shift=(12, -8), axis=(0, 1))

# Estimate the shift needed to align moving with fixed.
estimated_shift, _, _ = phase_cross_correlation(fixed, moving)
aligned = shift(moving, estimated_shift, order=0, mode="grid-wrap")

imsave("registered.png", aligned)
print(estimated_shift)

The printed values describe the correction, while registered.png shows the aligned image. This is a translation-only example; it does not address rotation, deformation, or images from different sensors. In a moving-camera video pipeline, Ultralytics YOLO tracking provides a related application: its camera-motion compensation estimates a frame-to-frame warp to help keep tracked object positions consistent. That tracker operation supports association across frames rather than producing a general-purpose registered image.