Flow Matching

Flow matching is a method for training generative models that turn random noise into data by following a learned velocity field. A neural network learns, for any point and any time between 0 and 1, the direction and speed in which that point should move, and generating a sample means starting from Gaussian noise and integrating that velocity with an ordinary differential equation (ODE) solver until time 1. Yaron Lipman and colleagues at Meta AI and the Weizmann Institute of Science introduced it in Flow Matching for Generative Modeling at ICLR 2023, and two concurrent papers arrived at nearly the same objective under the names rectified flow and stochastic interpolants. Flow matching trains continuous normalizing flows without simulating the ODE during training, covers diffusion models as a special case, and is the training objective behind Stable Diffusion 3, FLUX.1, Meta's Movie Gen, and many recent speech and robotics models.

How Flow Matching Works#

Training never generates anything. Each step draws a data sample x1, a noise sample x0 from a standard Gaussian, and a time t between 0 and 1, and places a point on the straight line between them: xt = (1 - t) * x0 + t * x1. A point on that line moves with the constant velocity x1 - x0, so the network v(xt, t) is trained with a plain mean squared error to predict x1 - x0. Each noise-data pair defines only one path, but many paths pass through any given point, and the velocity of the full noise-to-data flow at that point is the average of theirs. The paper proves that regressing on the simple per-pair targets gives the same gradients as regressing on that intractable average, so training is a plain regression loss function with no ODE simulation.

Generation integrates dx/dt = v(x, t) from t = 0 to t = 1, usually with Euler or higher-order solver steps. This PyTorch code trains a small velocity network on two clusters of 2D points and then samples with 1, 2, and 8 Euler steps:

import torch
from torch import nn

torch.manual_seed(0)
# Velocity network v(x, t) for 2D points; the data are two clusters at (2, 2) and (-2, -2)
net = nn.Sequential(nn.Linear(3, 128), nn.SiLU(), nn.Linear(128, 128), nn.SiLU(), nn.Linear(128, 2))
optimizer = torch.optim.Adam(net.parameters(), lr=1e-3)
centers = torch.tensor([[2.0, 2.0], [-2.0, -2.0]])

for step in range(2000):
    x1 = centers[torch.randint(0, 2, (256,))] + 0.2 * torch.randn(256, 2)  # data samples
    x0 = torch.randn(256, 2)  # noise samples
    t = torch.rand(256, 1)
    xt = (1 - t) * x0 + t * x1  # point on the straight path from noise to data
    loss = ((net(torch.cat([xt, t], dim=1)) - (x1 - x0)) ** 2).mean()  # regress the path's velocity
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

# Generate: start from noise and follow the learned velocity field with Euler steps
for steps in (1, 2, 8):
    x = torch.randn(1000, 2)
    with torch.no_grad():
        for i in range(steps):
            x = x + net(torch.cat([x, torch.full((1000, 1), i / steps)], dim=1)) / steps
    near = (torch.cdist(x, centers).min(dim=1).values < 0.6).float().mean()
    print(f"{steps}-step sampling: {near:.0%} of samples land near a cluster")

It prints 1-step sampling: 0% of samples land near a cluster, then 47% for 2 steps and 94% for 8 steps. At t = 0 every noise point has the same expected destination, the mean of the data, so a single step sends all samples to the empty space between the two clusters. Each training path is straight, but paths that start near each other end in different clusters, and the learned flow that averages them still curves. Few-step generators therefore add techniques such as reflow, which retrains on the model's own noise-to-sample pairs to straighten the flow, or distillation into consistency models.

Flow Matching vs. Related Methods#

• Diffusion models: Diffusion training corrupts data along a noise schedule and teaches a network to predict the added noise, then samples by reversing that process. The flow matching paper showed that its objective accepts any Gaussian probability path, including the paths diffusion models use, and found it a more stable way to train them. The straight path is a separate, simpler choice. Both families share architectures such as the diffusion transformer (DiT), classifier-free guidance, and ODE samplers.
• Rectified flow: Uses the same straight-line interpolation and velocity loss, so for the linear path the two names are largely synonyms. The rectified flow paper adds the reflow procedure to straighten trajectories further.
• Normalizing flows: Discrete normalizing flows stack invertible layers and train by exact likelihood. Continuous normalizing flows, a form of neural ODE, define the transformation with an ODE but were trained by maximum likelihood, which solves the ODE at every training step. Flow matching trains the same kind of ODE model by regression, without that solve.
• Optical flow: Unrelated despite the name. Optical flow is the apparent motion of pixels between video frames, while flow matching moves samples through the space of possible data.

Where Flow Matching Is Used#

• Image generation: Stability AI trained Stable Diffusion 3 with a rectified flow transformer and sampled training timesteps from a logit-normal distribution that favors intermediate noise levels, and Black Forest Labs' FLUX.1 is a 12-billion-parameter rectified flow transformer. Images from these text-to-image models serve as synthetic data that adds rare objects, weather, and viewpoints to datasets for training YOLO26 detection models.
• Video generation: Meta's Movie Gen is a 30-billion-parameter transformer trained with flow matching that generates 1080p video up to 16 seconds long at 16 frames per second.
• Speech: Meta's Voicebox trained a non-autoregressive flow matching model on more than 50,000 hours of speech, and in zero-shot text-to-speech it reached a 1.9% word error rate against 5.9% for VALL-E while running up to 20 times faster. F5-TTS later combined flow matching with a diffusion transformer.
• Robot control: Physical Intelligence's π0 adds a 300-million-parameter flow matching "action expert" to a 3-billion-parameter vision-language model, producing chunks of 50 future actions with 10 integration steps and controlling robots at up to 50 Hz.