Stereo Vision
Stereo vision measures depth by matching each point between two offset cameras and triangulating the shift between views, as in robots, cars, and rovers.
Stereo vision is the computer vision technique of recovering depth from two images of the same scene taken by cameras a known distance apart. A point on a nearby object lands at clearly different positions in the left and right images, while a distant point lands at almost the same position, and that horizontal offset, called disparity, converts directly into distance by triangulation. Human binocular vision works the same way with eyes about 6 cm apart. Stereo camera rigs give robotics systems, cars, drones, and planetary rovers dense depth estimation from ordinary passive cameras.
How Stereo Vision Works#
A stereo system first needs camera calibration: the focal length and lens distortion of each camera, plus the rotation and translation between the two. Rectification then warps both images so that epipolar geometry takes its simplest form, in which every point's match lies on the same row of the other image. That turns the correspondence problem, finding the same scene point in both views, into a one-dimensional search along a row. For a matched pixel with disparity d, depth follows from Z = f × B / d, where f is the focal length in pixels and B is the baseline, the distance between the cameras.
This example runs OpenCV's semi-global matcher on a rectified pair from the Middlebury 2014 stereo benchmark, which ships with scikit-image together with subpixel-accurate ground-truth disparity measured with structured light:
import cv2
import numpy as np
from skimage import data
# Rectified Middlebury stereo pair with ground-truth disparity (focal length 995 px, baseline 0.193 m)
left, right, true_disp = data.stereo_motorcycle()
left, right = cv2.cvtColor(left, cv2.COLOR_RGB2GRAY), cv2.cvtColor(right, cv2.COLOR_RGB2GRAY)
# Semi-global matching searches each pixel's row in the other image; OpenCV returns 1/16-pixel units
sgbm = cv2.StereoSGBM_create(minDisparity=0, numDisparities=80, blockSize=5, P1=200, P2=800, uniquenessRatio=10)
disp = sgbm.compute(left, right).astype(np.float32) / 16
# Score the pixels that have ground truth and a confident match
valid = np.isfinite(true_disp) & (disp > 0)
good = np.abs(disp - true_disp)[valid] < 1
print(f"Matched {valid.mean():.0%} of pixels, {good.mean():.1%} of them within 1 px of ground truth")
# Triangulate depth: Z = focal * baseline / (disparity + offset between the two principal points)
depth, true_depth = (994.978 * 0.193 / (d + 31.086) for d in (disp, true_disp))
print(f"Depth at the image center: {depth[250, 370]:.2f} m (ground truth {true_depth[250, 370]:.2f} m)")It prints Matched 79% of pixels, 91.0% of them within 1 px of ground truth and a depth of 2.40 m at the image center, equal to the ground truth. Most unmatched pixels form the 80-pixel strip along the left edge, where the search over 80 disparities would run past the border of the right image.
Matching Algorithms and Accuracy#
Comparing small windows of pixels fails where a window has no texture, repeats a pattern, or straddles an object edge. Semi-global matching, introduced by Heiko Hirschmüller in 2005 and implemented in modified form as OpenCV's StereoSGBM, adds a smoothness penalty for disparity jumps and aggregates it along several 1D paths across the image, which fills in weakly textured regions while keeping object edges sharp. Deep learning methods replace hand-built costs with learned feature maps and a cost volume over candidate disparities, and RAFT-Stereo refines a disparity field with recurrent updates in the style of optical flow networks. Methods are compared on the Middlebury stereo benchmark, started by Daniel Scharstein and Richard Szeliski in 2002, and on KITTI driving scenes with LiDAR ground truth.
Precision falls quickly with distance. Differentiating the depth formula gives a depth error of about Z² / (f × B) per pixel of disparity error, so with the camera above a one-pixel error means 3 cm at 2.4 m and 52 cm at 10 m. A wider baseline or longer focal length extends the useful range at the cost of a larger minimum distance and more regions seen by only one camera. Active stereo cameras such as the Intel RealSense D400 series project an infrared dot pattern onto the scene, which gives blank walls the texture that matching needs.
Stereo Vision vs. Monocular Depth and Active Sensors#
Monocular depth estimation predicts depth from a single image using learned cues such as perspective, object size, and occlusion, so its scale comes from training data, while stereo measures scale geometrically from the baseline. Ultralytics YOLO26 depth estimation models take the monocular route, predicting a depth map in meters from one RGB image, and model.calibrate() fits their metric scale to a specific camera from a small set of images with measured depth. LiDAR and time-of-flight cameras measure distance actively by timing emitted light, and many vehicles combine them with stereo through sensor fusion. Stereo depth also feeds 3D object detection, 3D reconstruction, and visual SLAM, and adding time turns it into scene flow, the 3D motion of every point.
Real-World Applications#
- Driver assistance: Subaru's EyeSight system uses two cameras mounted near the rearview mirror to measure the distance to vehicles and pedestrians ahead for adaptive cruise control and automatic emergency braking.
- Robot picking: Stereo depth cameras on robot arms locate parts in bins and on conveyors, giving the 3D position and orientation a gripper needs.
- Drones: Consumer drones such as DJI's Mavic series detect obstacles with binocular vision sensors facing several directions.
- Planetary rovers: NASA's Perseverance rover drives itself across Mars by building 3D terrain maps on board from its stereo navigation cameras.










