Depth Bleeding

Depth bleeding is an artifact in depth maps in which depth values spill across the boundary between a foreground object and its background, so pixels along the edge take the other surface's depth or a blend of both. It appears at depth discontinuities, the occlusion boundaries where one surface ends and a much farther one begins, and it affects every source of depth: stereo vision, time-of-flight and LiDAR sensors, and learned depth estimation. The edges matter far more than their pixel count suggests. Back-projected into a 3D point cloud, bled pixels become flying pixels, points hanging in empty space between an object and the wall behind it. In augmented reality they make virtual objects poke through or float in front of real edges, in novel view synthesis they stretch edges like rubber sheets, and for robotics and autonomous vehicles they make objects look wider than they are, which misplaces a robot's grasp and stretches the boxes of 3D object detection.

Where Depth Bleeding Comes From#

In stereo matching, each pixel's match is decided by comparing a window of neighbors, and near an edge that window covers both surfaces. Foreground pixels in the window can dominate the matching cost, so the foreground's depth spreads onto background pixels next to it, an effect the stereo literature calls foreground fattening. Pixels visible to only one camera have no correct match at all and pick up a neighbor's depth.

Neural networks for monocular depth estimation produce bleeding in two ways. Their encoders downsample the image into coarse feature maps, so many decoders predict depth at a fraction of the input resolution and interpolate up, which blends the two surfaces across every edge. And where a boundary pixel could belong to either surface, a squared-error loss function is minimized by predicting the average of the two depths, a value that lies on neither surface. Ultralytics YOLO26 depth models predict log-depth at a quarter of the input resolution, and the predictor upsamples the map bilinearly to the original image size, so a jump between two surfaces in the output becomes a ramp a few pixels wide.

Sensors bleed physically. A time-of-flight pixel or a LiDAR beam whose footprint straddles an edge receives light from both surfaces and reports a depth in between, the classic source of flying pixels in depth-camera point clouds. RGB-D cameras add misalignment, since depth is measured from a slightly different position than color and edges shift when one is reprojected onto the other.

This example upsamples a quarter-resolution depth map of a person standing 1.5 m in front of a wall 4 m away, the way many depth networks reach full resolution, and counts pixels whose depth belongs to neither surface:

import torch
import torch.nn.functional as F

# A depth map predicted at 1/4 resolution: a person 1.5 m away in front of a wall 4 m away
low = torch.full((1, 1, 120, 160), 4.0)
low[..., 30:, 60:100] = 1.5

for mode in ["bilinear", "nearest"]:
    # Upsample 4x to the 480 x 640 image
    depth = F.interpolate(low, size=(480, 640), mode=mode)[0, 0]

    # Bleeding pixels hold depths that belong to neither surface
    bleeding = (depth > 1.6) & (depth < 3.9)
    print(f"{mode}: {bleeding.sum().item()} bleeding pixels, row 300 across the edge: {depth[300, 394:404].tolist()}")

Bilinear upsampling creates 3,518 bleeding pixels, a ramp of 1.81, 2.44, 3.06, and 3.69 m along every edge of the silhouette, each of which would float in midair in a point cloud. Nearest-neighbor upsampling creates none, because it never invents new values, but in real images the true edge falls inside a coarse pixel, so nearest-neighbor output places edges up to two pixels off and turns diagonal edges into staircases.

Reducing and Measuring Depth Bleeding#

Edge-aware upsampling, a form of guided depth super-resolution, uses the full-resolution color image as a guide. Joint bilateral upsampling, introduced by Johannes Kopf and colleagues in 2007, and the guided filter weight each neighbor by color similarity, so depth stops at image edges. The opposite failure, texture copying, follows when color edges without a depth change, such as print on a box, are carved into the depth map. Networks reduce bleeding with higher-resolution outputs, boundary-aware training losses, and refinement guided by image segmentation, and Apple's Depth Pro was designed and evaluated specifically for sharp boundaries. For stereo, a left-right consistency check marks pixels whose matches disagree between the two directions, and for any source, flying pixels can be removed by discarding points where depth jumps sharply between neighbors or where the surface is nearly parallel to the viewing ray.

Standard metrics barely register bleeding, since edge pixels are a small share of an image and average errors such as absolute relative error hardly move. The iBims-1 benchmark, released by Tobias Koch and colleagues at the Technical University of Munich in 2018, scores depth edges directly with depth boundary errors: accuracy measures how far predicted edges lie from manually annotated ones, and completeness measures how many true edges a prediction misses.