License YOLO
Back to Ultralytics Glossary

Latent Space

A latent space is the compressed space of learned features where a neural network places similar inputs near each other, used for search and generation.

A latent space is the space of internal representations a machine learning model learns, where each input becomes a point, usually with far fewer dimensions than the raw data, and inputs that are similar in meaning sit close together. "Latent" means hidden: the coordinates are not measured in the data but inferred by the model during training. A 640 x 640 color photo holds 1,228,800 pixel values, an image model can summarize it as a vector of a few hundred numbers, and in a well-trained space the nearest neighbors of a photo of a car are other cars, while directions in the space correspond to properties such as pose, color, or style.

Latent spaces work because natural data is highly redundant. Most possible pixel arrays look like noise, and real images occupy a thin, curved region of pixel space, an idea known as the manifold hypothesis. A neural network that learns coordinates along that region can describe each image with a handful of numbers.

Latent Space, Embeddings, and Feature Maps#

An embedding is a single point in a latent space, the vector for one image, word, or user, and the latent space is the set of all such points together with the distances between them; "embedding space" is a synonym. Feature maps inside a convolutional network are spatial latent representations, with a vector at every location, which a model often pools into one embedding per image. Dimensionality reduction methods also produce latent spaces: principal component analysis (PCA) finds a linear one, and t-SNE is mostly used to project a learned latent space down to two dimensions for plotting.

How Models Learn a Latent Space#

An autoencoder learns one directly. Its encoder compresses each input into a short vector, its decoder reconstructs the input from that vector, and minimizing the reconstruction error forces the bottleneck to keep what distinguishes one input from another. No labels are involved, yet the resulting space groups similar inputs, which a k-nearest neighbors (KNN) classifier can measure:

import torch
from sklearn.datasets import load_digits
from sklearn.neighbors import KNeighborsClassifier
from torch import nn

torch.manual_seed(0)
images, labels = load_digits(return_X_y=True)  # 1,797 handwritten digits, 8 x 8 = 64 pixels each
x = torch.tensor(images / 16, dtype=torch.float32)

# An autoencoder squeezes 64 pixels through an 8-number bottleneck: the latent space
encoder = nn.Sequential(nn.Linear(64, 32), nn.ReLU(), nn.Linear(32, 8))
decoder = nn.Sequential(nn.Linear(8, 32), nn.ReLU(), nn.Linear(32, 64))
optimizer = torch.optim.Adam([*encoder.parameters(), *decoder.parameters()], lr=0.01)
for _ in range(1000):
    loss = (decoder(encoder(x)) - x).square().mean()  # reconstruction error, no labels used
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

# Classify held-out digits by their nearest neighbors in each space
codes = encoder(x).detach().numpy()
for name, features in (("64 pixels", images), ("8 latent values", codes)):
    knn = KNeighborsClassifier().fit(features[:1400], labels[:1400])
    print(f"{name}: {knn.score(features[1400:], labels[1400:]):.2f} accuracy")

It prints 64 pixels: 0.97 accuracy and 8 latent values: 0.93 accuracy. Eight learned numbers per image keep enough of the digit's identity for nearest-neighbor classification to stay close to what all 64 pixels achieve, although the autoencoder never saw a label.

Variational autoencoders, introduced by Diederik Kingma and Max Welling in 2013, encode each input as a distribution instead of a point, which makes the space smooth enough to sample new data from it and to interpolate between examples. Contrastive learning shapes a space by pulling matching pairs together and pushing other pairs apart; CLIP trains on image-caption pairs so that images and text share one latent space. Supervised networks learn a latent space as a side effect of their task. model.embed() in Ultralytics YOLO26 runs predict mode and returns one vector per image by averaging the feature map of the layer before the detection head over all spatial positions, a representation suited to clustering and similarity search.

What Latent Spaces Enable#

Generation is cheaper in a compact space. Latent diffusion models run the denoising process of diffusion models on latents instead of pixels. Stable Diffusion, built on the latent diffusion paper, uses an autoencoder to compress a 512 x 512 RGB image into a 64 x 64 x 4 latent, 48 times fewer values, which makes high-resolution generation affordable. Generative adversarial networks (GANs) map random latent vectors to images, so walking between two latent points morphs one generated face into another.

Search and recognition work by measuring distance. Semantic search systems embed documents or images once, store the vectors in a vector database, and return the items with the highest cosine similarity to a query's embedding. In a shared image-text space such as CLIP's, zero-shot learning classifies an image by the class name whose text embedding lies closest, with no training images for that class. For anomaly detection, an autoencoder trained only on normal parts reconstructs defective parts poorly, and inputs whose embeddings fall far from the normal cluster are flagged for inspection.

Directions in a well-organized space encode attributes. In the word2vec embeddings that Tomas Mikolov and colleagues released in 2013, a staple of natural language processing (NLP), the vector for "king" minus "man" plus "woman" lands nearest to "queen".

Explore solutions

Computer vision in sports

Computer vision in sports

Discover how Ultralytics YOLO powers sports vision with player and ball tracking, performance analytics, coaching insights, and athlete rehabilitation.
Learn more
Computer vision for aerial imagery

Computer vision for aerial imagery

Turn drone and aerial imagery into real-time insights for agriculture, aquaculture, conservation, and geospatial analysis with Ultralytics YOLO.
Learn more
Computer vision in aerospace

Computer vision in aerospace

Apply computer vision in aerospace with Ultralytics YOLO for aircraft inspection, runway FOD detection, airport ground operations, and composite defect detection.
Learn more
Computer vision in security

Computer vision in security

Protect every site with Ultralytics YOLO models. Vision AI powers perimeter monitoring, threat detection, license-plate recognition, and workplace safety.
Learn more
Computer vision in robotics

Computer vision in robotics

Power smarter machines with Ultralytics YOLO models. Vision AI in robotics drives autonomous navigation, perception, object tracking, and real-time control.
Learn more
Computer vision in logistics

Computer vision in logistics

Streamline logistics with Ultralytics YOLO models. Vision AI enables package inspection, sorting, vehicle tracking, and real-time warehouse safety monitoring.
Learn more
Computer vision in retail

Computer vision in retail

Reimagine retail with Ultralytics YOLO models. Vision AI powers inventory tracking, shelf monitoring, queue management, and smarter customer insights.
Learn more
Computer vision in healthcare

Computer vision in healthcare

Build healthcare solutions with Ultralytics YOLO models. Vision AI in healthcare powers faster medical imaging, smarter diagnostics, and patient monitoring.
Learn more
Computer vision in manufacturing

Computer vision in manufacturing

Optimize manufacturing with Ultralytics YOLO models. Vision AI drives quality control, defect detection, PPE compliance, and assembly line automation.
Learn more
Computer vision in automotive

Computer vision in automotive

Apply computer vision in automotive with Ultralytics YOLO models. Vision AI elevates road safety, driver assistance, and vehicle automation for smarter roads.
Learn more
Computer vision in agriculture

Computer vision in agriculture

Bring vision AI to smart agriculture with Ultralytics YOLO models. Power crop monitoring, livestock tracking, and precision farming for higher, smarter yields.
Learn more

Let's build the future of AI together!

Begin your journey with the future of machine learning