Kalman Filter (KF)

A Kalman filter (KF) is a recursive algorithm that estimates the hidden state of a changing system, such as an object's position and velocity, from a stream of noisy measurements. At every time step it predicts the next state from a model of how the system moves, then corrects that prediction with the new measurement, weighting each by how uncertain it is. Rudolf E. Kálmán published the method in 1960, and for linear systems with Gaussian noise it gives the minimum mean squared error estimate. It runs in GPS receivers, aircraft and spacecraft navigation, robot localization, and nearly every multi-object tracker in computer vision, where it smooths jittery detections and predicts where each object will appear next.

How the Kalman Filter Works#

The filter keeps two quantities: a state estimate x, for example position and velocity, and a covariance matrix P that describes how uncertain that estimate is. Each cycle has two steps:

1. Predict: A motion model F projects the state forward, x = F x, and the uncertainty grows by the process noise Q, which covers motion the model does not capture: P = F P Fᵀ + Q.
2. Update: A new measurement z arrives, related to the state by the measurement model H. The filter computes the residual z − H x and the Kalman gain K = P Hᵀ (H P Hᵀ + R)⁻¹, where R is the measurement noise. The corrected state is x + K (z − H x), and the uncertainty shrinks to (I − K H) P.

The Kalman gain sets the balance between model and sensor. When measurements are precise relative to the prediction, the gain approaches 1 and the estimate follows the measurement; when they are noisy, the gain approaches 0 and the estimate follows the prediction. Because the state can include quantities that are never measured directly, the filter also infers them: a tracker that observes only positions still estimates velocity. When a measurement is missing, the filter skips the update and coasts on its prediction:

Kalman Filters in Object Tracking#

In object tracking, each track carries its own Kalman filter. The state holds the bounding box center and size plus their velocities, and a constant-velocity model predicts where the box will be in the next frame. Those predictions are matched to new detections by Intersection over Union (IoU), and each matched detection becomes the measurement that updates its track. When an object is hidden for a few frames, the track keeps moving along its predicted path, so the detection that reappears lands close enough to its prediction to keep the old ID.

Every tracker in Ultralytics YOLO track mode works this way: ByteTrack's KalmanFilterXYAH tracks an 8-dimensional state of box center, aspect ratio, height, and their velocities, BoT-SORT and TrackTrack use a width-and-height variant, and both scale their noise with box size so that large, nearby objects tolerate larger jumps in pixels. The same predict and update steps fit in a few lines of NumPy for one coordinate of a box center:

import numpy as np

F = np.array([[1.0, 1.0], [0.0, 1.0]])  # constant-velocity model, state = [position, velocity]
H = np.array([[1.0, 0.0]])  # the detector measures position only
Q = 0.001 * np.eye(2)  # process noise: how far real motion may stray from the model
R = np.array([[25.0]])  # measurement noise: detections jitter by about 5 px

rng = np.random.default_rng(0)
truth = 100 + 3.0 * np.arange(60)  # box center moving 3 px per frame
z = truth + rng.normal(0, 5.0, truth.size)  # noisy detections

x, P = np.array([z[0], 0.0]), np.diag([25.0, 100.0])  # start from the first detection
estimates = [x[0]]
for measurement in z[1:]:
    x, P = F @ x, F @ P @ F.T + Q  # predict
    K = P @ H.T @ np.linalg.inv(H @ P @ H.T + R)  # Kalman gain
    x = x + K @ (measurement - H @ x)  # update with the weighted residual
    P = (np.eye(2) - K @ H) @ P
    estimates.append(x[0])

print(f"Detection error: {np.sqrt(np.mean((z - truth) ** 2)):.2f} px")
print(f"Kalman error: {np.sqrt(np.mean((np.array(estimates) - truth) ** 2)):.2f} px")
print(f"Estimated velocity: {x[1]:.2f} px/frame")

It prints a root mean squared error of 4.50 px for the raw detections and 2.48 px for the filtered estimates, and an estimated velocity of 2.98 px per frame, close to the true 3 px, even though velocity was never measured.

Related Filters#

The standard Kalman filter assumes linear motion and measurement models with Gaussian noise. Its relatives relax those assumptions:

• Extended Kalman filter (EKF): Linearizes nonlinear models around the current estimate with Jacobians at every step, the standard choice for GPS and inertial navigation.
• Unscented Kalman filter (UKF): Passes a small set of carefully chosen sample points through the nonlinear model, which is usually more accurate than linearization and needs no derivatives.
• Particle filter: Represents the state distribution with many weighted random samples, so it handles non-Gaussian and multimodal beliefs, such as a robot unsure which of two corridors it is in, at a much higher compute cost.
• Hidden Markov model (HMM): Uses the same predict-and-update recursion for discrete states; the Kalman filter is its counterpart for continuous, linear-Gaussian states.

Applications Beyond Tracking#

• Robotics and sensor fusion: Robots combine wheel odometry, IMUs, lidar, and cameras into one pose estimate, and EKF-based methods are a classic approach to simultaneous localization and mapping.
• Augmented reality: Headsets filter head-pose measurements to keep virtual objects steady with low latency.
• Economics and signal processing: Analysts use Kalman smoothing to estimate unobserved trends from noisy time series.
• Navigation: Aircraft, ships, drones, and phones fuse GPS with inertial sensors so position stays smooth between satellite fixes. Navigation was also the filter's first major application: in the early 1960s, Stanley F. Schmidt at NASA Ames adapted it to nonlinear trajectory estimation for the Apollo navigation computer.