Template Matching
Template matching is a classical computer vision technique that finds a small reference patch in a larger image by scoring every position.
Template matching is a classical computer vision technique that finds where a small reference image, the template, appears inside a larger image. The template is slid across the image one pixel at a time, a similarity score is computed at every position, and the best-scoring position is reported as the match. The output is a score map. Its maximum marks the match for correlation measures, and its minimum marks it for squared-difference measures.
Because it compares raw pixel values and learns nothing from data, template matching is closer to image processing than to machine learning. It remains a standard baseline in libraries such as OpenCV, which exposes it as cv2.matchTemplate.
How the Scoring Works#
At each offset, the template is compared with the image patch of the same size beneath it. The sliding comparison is closely related to the cross-correlation used in convolution layers, with a fixed similarity measure in place of a learned filter. The common measures are:
- Squared difference: Sums the squared pixel differences, so lower is better. It is cheap but shifts with any change in brightness.
- Cross-correlation: Multiplies template and patch pixels, so a bright patch can outscore an exact match.
- Normalized cross-correlation (NCC): Divides by the patch and template magnitudes, which fixes the bright-patch problem and bounds the score.
- Correlation coefficient: Subtracts the mean of each patch before normalizing, so a uniform brightness offset leaves the score unchanged. J. P. Lewis's 1995 Vision Interface paper on fast template matching, expanded as Fast Normalized Cross-Correlation, showed that this normalized form can be computed efficiently with precomputed integral tables of the image and its square. scikit-image offers the same idea through its match_template example.
After scoring, a threshold keeps only strong candidates. When several copies of the template appear, such as repeated coins or tiles, each local extremum past the threshold is a candidate detection. Overlapping hits are typically merged with non-maximum suppression. The scores are similarity values rather than calibrated confidences.
The snippet below crops a patch from the sample bus photo, adds a uniform brightness offset to the scene, and recovers the patch location with the correlation coefficient.
import cv2
import numpy as np
from ultralytics.utils import ASSETS
# Load the sample image in grayscale as float32
img = cv2.imread(str(ASSETS / "bus.jpg"), cv2.IMREAD_GRAYSCALE).astype(np.float32)
# Crop a 200 x 200 template whose top-left corner is at x=200, y=400
template = img[400:600, 200:400].copy()
# Add a uniform brightness offset to the scene (no clipping in float32)
scene = img + 40
# Score every position and take the peak of the score map
scores = cv2.matchTemplate(scene, template, cv2.TM_CCOEFF_NORMED)
_, best_score, _, best_xy = cv2.minMaxLoc(scores)
print(f"Best match at {best_xy} with score {best_score:.3f}")The mean-subtracted score ignores a uniform offset, so the peak sits at the crop corner (200, 400) with a score of 1.0 up to rounding.
Where It Is Used#
Template matching works when the target looks the same every time:
- Game and GUI automation: OpenCV's own tutorial finds every coin in a game screenshot by thresholding the score map, since rendered sprites repeat pixel for pixel.
- Visual inspection: A reference crop of a label is searched for in each frame from a fixed camera to check that it is present, a basic machine vision task.
- Motion and stereo matching: Small blocks are matched between consecutive frames or between left and right views to estimate displacement or disparity.
- Medical imaging: An NCC study in PLOS ONE selects a template in one image and slides it over a second image of the same patient taken at a different time to follow internal motion. The same idea underlies image registration.
Limits and Alternatives#
A template captures one exact appearance, so the score drops when the object is rotated, resized, partly hidden, or seen against a new background. A direct exhaustive search costs roughly image size times template size. Matching over an image pyramid handles scale and rotating the template handles angle, but each extra scale or angle adds another full search, while narrowing the search region reduces the work.
Other methods trade exactness for invariance. Keypoint approaches such as SIFT match local descriptors that survive scale and rotation, and the broader family is covered under image matching. The Hough transform finds parametric shapes such as lines and circles. Learned object detection models learn the appearance of a whole class from labeled examples, so one model covers many variations of an object where template matching needs a template per variation.
Ultralytics YOLO26 takes the learned route. A model trained on annotated images in Ultralytics Platform predicts bounding boxes for the classes it was trained on, and data augmentation during training can add rotation, scale, and brightness variations.
OpenCV's matchTemplate offers six comparison methods: squared difference, cross-correlation, and correlation coefficient, each with a normalized variant.










