License YOLO
Back to Ultralytics Glossary

Latent Consistency Models (LCMs)

Latent consistency models (LCMs) distill Stable Diffusion into a model that makes images in one to four steps. Covers distillation, LCM-LoRA, and speed.

A latent consistency model (LCM) is a text-to-image model distilled from a latent diffusion model such as Stable Diffusion so that it generates an image in one to four sampling steps instead of the usual 20 to 50. It applies the consistency model idea, a network that maps any noisy point on the denoising path straight to its clean end, inside the compressed latent space of the original model. The text encoder and image decoder are reused unchanged, and the student starts from the teacher's U-Net weights. Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao of Tsinghua University introduced LCMs in October 2023 in Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference, reporting that a 768 x 768 model sampling in 2 to 4 steps took only 32 A100 GPU hours to train.

How Latent Consistency Distillation Works#

A standard diffusion model generates by solving a probability flow ordinary differential equation (ODE) one small step at a time, from a pure-noise latent to a clean latent that the autoencoder's decoder turns into pixels. Latent consistency distillation trains a student copy of the U-Net to make that whole journey in one call. In each training step, the frozen teacher takes one ODE-solver step along the trajectory, and the student must predict the same clean latent from both ends of that step:

Repeated across random timesteps, this pulls every point on a trajectory toward the same output. Two changes make it work for text-to-image models. The teacher's step already includes classifier-free guidance at a scale w, and w becomes an input to the student, so guided images cost one forward pass instead of two. A skipping-step schedule has the teacher jump k timesteps at once instead of one, which speeds up convergence. A teacher-free variant, latent consistency fine-tuning, adapts an existing LCM to a custom dataset.

Consistency distillation is a form of knowledge distillation, the teacher-student pattern that Ultralytics YOLO26 training also supports through its distill_model argument: training yolo26n.pt with distill_model="yolo26s.pt" makes the nano student match the larger model's feature maps, which raises the student's mAP on COCO at no added inference cost.

Sampling in Four Steps#

An LCM samples by alternating jumps and re-noising. It predicts the clean latent from pure noise, adds fresh noise to bring that prediction back to a smaller timestep, predicts again, and decodes the last prediction. The LCMScheduler in Hugging Face diffusers picks those timesteps from Stable Diffusion's 1,000-step training schedule:

from diffusers import DDIMScheduler, LCMScheduler

# Stable Diffusion's 1,000-step training noise schedule
config = {"num_train_timesteps": 1000, "beta_start": 0.00085, "beta_end": 0.012, "beta_schedule": "scaled_linear"}
ddim = DDIMScheduler(**config)
lcm = LCMScheduler(**config)

# A standard sampler visits 50 noise levels; an LCM jumps to a clean latent from each of 4
ddim.set_timesteps(50)
lcm.set_timesteps(4)
print(f"DDIM: {len(ddim.timesteps)} steps, starting {ddim.timesteps[:4].tolist()} ...")
print(f"LCM:  {len(lcm.timesteps)} steps, {lcm.timesteps.tolist()}")

The DDIM sampler walks down from timestep 980 in steps of 20, one U-Net call per step, while the LCM visits only timesteps 999, 759, 499, and 259. Each extra step lets the model correct details of the previous prediction, so quality rises from one step to four.

LCM-LoRA and Real-Time Uses#

A month after the original paper, the authors and Hugging Face engineers released LCM-LoRA, which stores the distillation as a LoRA adapter instead of a full set of new weights. They trained it for Stable Diffusion v1.5, SSD-1B, and SDXL, and it plugs into fine-tuned checkpoints and style LoRAs of the same base model without further training. Few-step sampling made real-time inference possible for interactive tools: the open-source StreamDiffusion pipeline ran LCM-LoRA with a KohakuV2 checkpoint at 4 denoising steps and 38 frames per second for text-to-image on one RTX 4090 GPU.

  • Consistency models: OpenAI's original formulation, which works directly on pixels and was benchmarked on CIFAR-10 and ImageNet 64 x 64. LCMs carry it into a latent space and add text conditioning and guidance.
  • Adversarial diffusion distillation: Stability AI's SDXL Turbo, also from November 2023, reaches one to four steps by adding a discriminator to the distillation loss.
  • Large concept models: Meta's large concept models share the LCM abbreviation but are language models that predict whole-sentence embeddings, with no relation to image generation.
  • Rectified flow: Trains straighter noise-to-data paths so that a sampler needs fewer steps, the approach behind Stable Diffusion 3 and FLUX.

Explore solutions

Computer vision in sports

Computer vision in sports

Discover how Ultralytics YOLO powers sports vision with player and ball tracking, performance analytics, coaching insights, and athlete rehabilitation.
Learn more
Computer vision for aerial imagery

Computer vision for aerial imagery

Turn drone and aerial imagery into real-time insights for agriculture, aquaculture, conservation, and geospatial analysis with Ultralytics YOLO.
Learn more
Computer vision in aerospace

Computer vision in aerospace

Apply computer vision in aerospace with Ultralytics YOLO for aircraft inspection, runway FOD detection, airport ground operations, and composite defect detection.
Learn more
Computer vision in security

Computer vision in security

Protect every site with Ultralytics YOLO models. Vision AI powers perimeter monitoring, threat detection, license-plate recognition, and workplace safety.
Learn more
Computer vision in robotics

Computer vision in robotics

Power smarter machines with Ultralytics YOLO models. Vision AI in robotics drives autonomous navigation, perception, object tracking, and real-time control.
Learn more
Computer vision in logistics

Computer vision in logistics

Streamline logistics with Ultralytics YOLO models. Vision AI enables package inspection, sorting, vehicle tracking, and real-time warehouse safety monitoring.
Learn more
Computer vision in retail

Computer vision in retail

Reimagine retail with Ultralytics YOLO models. Vision AI powers inventory tracking, shelf monitoring, queue management, and smarter customer insights.
Learn more
Computer vision in healthcare

Computer vision in healthcare

Build healthcare solutions with Ultralytics YOLO models. Vision AI in healthcare powers faster medical imaging, smarter diagnostics, and patient monitoring.
Learn more
Computer vision in manufacturing

Computer vision in manufacturing

Optimize manufacturing with Ultralytics YOLO models. Vision AI drives quality control, defect detection, PPE compliance, and assembly line automation.
Learn more
Computer vision in automotive

Computer vision in automotive

Apply computer vision in automotive with Ultralytics YOLO models. Vision AI elevates road safety, driver assistance, and vehicle automation for smarter roads.
Learn more
Computer vision in agriculture

Computer vision in agriculture

Bring vision AI to smart agriculture with Ultralytics YOLO models. Power crop monitoring, livestock tracking, and precision farming for higher, smarter yields.
Learn more

Let's build the future of AI together!

Begin your journey with the future of machine learning