Latent Consistency Models (LCMs)
Latent consistency models (LCMs) distill Stable Diffusion into a model that makes images in one to four steps. Covers distillation, LCM-LoRA, and speed.
A latent consistency model (LCM) is a text-to-image model distilled from a latent diffusion model such as Stable Diffusion so that it generates an image in one to four sampling steps instead of the usual 20 to 50. It applies the consistency model idea, a network that maps any noisy point on the denoising path straight to its clean end, inside the compressed latent space of the original model. The text encoder and image decoder are reused unchanged, and the student starts from the teacher's U-Net weights. Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao of Tsinghua University introduced LCMs in October 2023 in Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference, reporting that a 768 x 768 model sampling in 2 to 4 steps took only 32 A100 GPU hours to train.
How Latent Consistency Distillation Works#
A standard diffusion model generates by solving a probability flow ordinary differential equation (ODE) one small step at a time, from a pure-noise latent to a clean latent that the autoencoder's decoder turns into pixels. Latent consistency distillation trains a student copy of the U-Net to make that whole journey in one call. In each training step, the frozen teacher takes one ODE-solver step along the trajectory, and the student must predict the same clean latent from both ends of that step:
Repeated across random timesteps, this pulls every point on a trajectory toward the same output. Two changes make it work for text-to-image models. The teacher's step already includes classifier-free guidance at a scale w, and w becomes an input to the student, so guided images cost one forward pass instead of two. A skipping-step schedule has the teacher jump k timesteps at once instead of one, which speeds up convergence. A teacher-free variant, latent consistency fine-tuning, adapts an existing LCM to a custom dataset.
Consistency distillation is a form of knowledge distillation, the teacher-student pattern that Ultralytics YOLO26 training also supports through its distill_model argument: training yolo26n.pt with distill_model="yolo26s.pt" makes the nano student match the larger model's feature maps, which raises the student's mAP on COCO at no added inference cost.
Sampling in Four Steps#
An LCM samples by alternating jumps and re-noising. It predicts the clean latent from pure noise, adds fresh noise to bring that prediction back to a smaller timestep, predicts again, and decodes the last prediction. The LCMScheduler in Hugging Face diffusers picks those timesteps from Stable Diffusion's 1,000-step training schedule:
from diffusers import DDIMScheduler, LCMScheduler
# Stable Diffusion's 1,000-step training noise schedule
config = {"num_train_timesteps": 1000, "beta_start": 0.00085, "beta_end": 0.012, "beta_schedule": "scaled_linear"}
ddim = DDIMScheduler(**config)
lcm = LCMScheduler(**config)
# A standard sampler visits 50 noise levels; an LCM jumps to a clean latent from each of 4
ddim.set_timesteps(50)
lcm.set_timesteps(4)
print(f"DDIM: {len(ddim.timesteps)} steps, starting {ddim.timesteps[:4].tolist()} ...")
print(f"LCM: {len(lcm.timesteps)} steps, {lcm.timesteps.tolist()}")The DDIM sampler walks down from timestep 980 in steps of 20, one U-Net call per step, while the LCM visits only timesteps 999, 759, 499, and 259. Each extra step lets the model correct details of the previous prediction, so quality rises from one step to four.
LCM-LoRA and Real-Time Uses#
A month after the original paper, the authors and Hugging Face engineers released LCM-LoRA, which stores the distillation as a LoRA adapter instead of a full set of new weights. They trained it for Stable Diffusion v1.5, SSD-1B, and SDXL, and it plugs into fine-tuned checkpoints and style LoRAs of the same base model without further training. Few-step sampling made real-time inference possible for interactive tools: the open-source StreamDiffusion pipeline ran LCM-LoRA with a KohakuV2 checkpoint at 4 denoising steps and 38 frames per second for text-to-image on one RTX 4090 GPU.
LCMs vs. Related Terms#
- Consistency models: OpenAI's original formulation, which works directly on pixels and was benchmarked on CIFAR-10 and ImageNet 64 x 64. LCMs carry it into a latent space and add text conditioning and guidance.
- Adversarial diffusion distillation: Stability AI's SDXL Turbo, also from November 2023, reaches one to four steps by adding a discriminator to the distillation loss.
- Large concept models: Meta's large concept models share the LCM abbreviation but are language models that predict whole-sentence embeddings, with no relation to image generation.
- Rectified flow: Trains straighter noise-to-data paths so that a sampler needs fewer steps, the approach behind Stable Diffusion 3 and FLUX.










