Fréchet Inception Distance (FID)

Fréchet Inception Distance (FID) is a metric for comparing images produced by a generative model with a reference set of real images. It converts both sets into learned visual features, summarizes each feature distribution, and measures the distance between them. A lower FID generally indicates that generated images are more similar to real images in both visual quality and diversity, while a higher FID suggests artifacts, missing variation, or a mismatch in content.

How FID Works#

FID evaluates image sets rather than individual image pairs. This makes it useful for assessing generative AI systems such as generative adversarial networks and diffusion models.

The calculation follows four main steps:

1. Real and generated images pass through an Inception v3 feature extractor, typically pretrained on ImageNet.
2. The network converts each image into an embedding, a numeric representation of its higher-level visual characteristics.
3. FID estimates the mean and covariance matrix of the real and generated feature sets. The mean represents their centers, while covariance describes how features vary together.
4. It combines the squared distance between the means with the difference between the covariance matrices. This computation includes a matrix square root.

The resulting FID score is nonnegative. Identical feature distributions would produce zero in the idealized limit, although finite samples and numerical effects mean even two real-image subsets can produce a nonzero score.

Interpreting an FID Score#

Lower is better, but there is no universal threshold for a “good” FID score. Interpretation depends on the dataset, image resolution, sample count, preprocessing pipeline, and feature extractor. Scores should only be compared when these conditions are consistent.

FID responds to two important aspects of generated imagery:

• Fidelity: Blurry images, unrealistic textures, distorted objects, and other visual artifacts can move generated features away from the real distribution.
• Diversity: Repetitive outputs or model collapse reduce feature variation, changing the generated distribution’s covariance.

However, FID does not explain why a score changed. It may also overlook rare but important failure cases, memorized images, incorrect prompt alignment, or dataset bias. Teams should therefore combine FID with visual inspection, application-specific tests, and subgroup analysis.

FID vs Related Metrics#

FID is often confused with other image-generation and computer vision metrics:

• Inception Score: Evaluates generated images without directly comparing them with real reference images. FID is generally more informative about distribution mismatch because it uses both real and generated samples.
• Kernel Inception Distance: Also compares feature distributions but uses a kernel-based estimator. It can be useful when sample-efficient or unbiased estimation is important.
• Mean average precision: Measures whether an object detector correctly classifies and localizes labeled objects. FID evaluates generated image distributions, not detection accuracy.
• Perceptual similarity: Usually compares corresponding image pairs. FID instead evaluates whether two collections have similar overall statistics.

Real-World Applications#

One practical application is evaluating synthetic data for manufacturing inspection. A team might generate images of scratches, cracks, or missing components and calculate FID against held-out photographs from a real production line. A high score can reveal that synthetic lighting, textures, or defect shapes do not resemble deployment conditions. After improving the generator, the team should still train its detector and use Ultralytics validation mode to verify task-specific performance.

A second example is simulated road-scene generation. Engineers may create nighttime, rain, or fog images to expand autonomous-driving training data. FID can compare each synthetic condition with real frames from that environment. A large distance warns of a domain mismatch that could cause a downstream detector to learn unrealistic backgrounds or weather artifacts rather than transferable road features.

Ultralytics Platform dataset management can help teams organize, inspect, and version real and synthetic image splits before training and deployment.

Calculating FID in Python#

The documented TorchMetrics FID implementation accepts batches of real and generated images:

import torch
from torchmetrics.image.fid import FrechetInceptionDistance

generator = torch.Generator().manual_seed(7)

real_images = torch.randint(0, 256, (64, 3, 64, 64), dtype=torch.uint8, generator=generator)
generated_images = torch.randint(32, 224, (64, 3, 64, 64), dtype=torch.uint8, generator=generator)

fid = FrechetInceptionDistance(feature=64)
fid.set_dtype(torch.float64)

fid.update(real_images, real=True)
fid.update(generated_images, real=False)

print(f"FID: {fid.compute().item():.3f}")

This small example uses 64-dimensional features for a quick demonstration. Standard benchmark comparisons typically use the default 2,048-dimensional representation and many more images. For reliable evaluation, keep preprocessing and sample counts fixed, report the exact configuration, repeat measurements when possible, and complement FID with the Ultralytics model evaluation workflow when generated data supports a downstream vision task.