Embeddings

An embedding is a learned vector of numbers that represents an item, such as a word, an image, a product, or a user, so that items with similar meaning or content get nearby vectors. Distance in this vector space becomes a measure of similarity, which lets software search, cluster, recommend, and classify by comparing vectors instead of raw pixels or strings. Embeddings are also the working representation inside neural networks: a large language model (LLM) converts every token into an embedding before its first transformer layer, and a vision model turns an image into feature vectors through its backbone. Embeddings used for search typically have a few hundred to a few thousand dimensions, far fewer than the raw input has values.

How Embeddings Are Created#

Embeddings come from two kinds of learned components:

• Lookup tables: For discrete items such as tokens, product IDs, or user IDs, an embedding layer stores one learnable row per item, and looking up an item returns its row. PyTorch's nn.Embedding is such a table. Its rows start random and are updated by backpropagation along with the rest of the model, so items that play similar roles in the training task drift toward similar vectors.
• Encoders: For images, audio, and free text, a network maps the whole input to one vector, often by pooling the feature maps of a CNN or reading a transformer's output tokens. Encoders trained for a task such as classification yield useful embeddings as a by-product, and encoders trained with contrastive learning shape the embedding space directly, as CLIP does by placing images and their captions in one shared space.

Word2vec, published by Tomas Mikolov and colleagues at Google in 2013, showed how much structure such training captures. In its word vectors, vector("King") − vector("Man") + vector("Woman") lands closest to vector("Queen"), and the pretrained Google News vectors it released cover 3 million words and phrases in 300 dimensions.

Comparing Embeddings#

Cosine similarity compares the directions of two vectors and ranges from −1 to 1. The dot product also accounts for their lengths, and Euclidean distance measures the straight-line gap between them. Many systems normalize embeddings to unit length, which makes cosine similarity and the dot product identical and ranks neighbors in the same order as Euclidean distance. YOLO26 models produce image embeddings through embed(), which skips the prediction head and returns one vector per image by average-pooling the output of the model's second-to-last layer:

import torch.nn.functional as F
from ultralytics import YOLO
from ultralytics.utils import ASSETS

# Load a YOLO26 classification model trained on ImageNet
model = YOLO("yolo26n-cls.pt")

# embed() returns one pooled feature vector per image instead of predictions
bus, zidane = model.embed([ASSETS / "bus.jpg", ASSETS / "zidane.jpg"])
print(f"Embedding length: {bus.shape[0]}")

# Higher cosine similarity means the model sees more similar content
print(f"Cosine similarity: {F.cosine_similarity(bus, zidane, dim=0).item():.3f}")

For YOLO26n-cls the embedding is a 256-dimensional vector, the pooled output of the attention block that ends its backbone, and the second line prints the cosine similarity of the two bundled sample photos. Running the same comparison against the stored embeddings of a whole collection is how an image similarity search ranks its results.

Searching Embeddings at Scale#

Comparing a query with millions of stored vectors one at a time is too slow, so vector databases build approximate nearest-neighbor indexes, such as graph-based HNSW indexes or inverted-file indexes, that find close matches while checking a small fraction of the collection. The collection is embedded once, and each query is embedded at search time with the same encoder, or with the paired text encoder when text searches images:

Storage shapes the design as much as accuracy does. Matryoshka representation learning trains embeddings whose leading dimensions work on their own, so vectors can be truncated to save space, and vector quantization compresses each vector into a few bytes. Hybrid search combines embedding similarity with keyword matching.

Applications#

• Semantic search and retrieval-augmented generation (RAG): Text embeddings retrieve passages by meaning, so a query about "car insurance claims" also finds documents that say "auto policy," and RAG passes the retrieved passages to an LLM as context.
• Recommendation systems: Users and items are embedded in one space, and the items nearest a user's vector become candidates to recommend.
• Zero-shot classification: Models with a shared image-text space classify an image by comparing its embedding with the embeddings of class names written as text.
• Object re-identification: Appearance embeddings link the same person or vehicle across cameras and across gaps in a track.
• Dataset exploration: Projecting embeddings to two dimensions with t-SNE or UMAP reveals clusters, near-duplicates, and outliers in a dataset, as the Clustering panel does for Ultralytics Platform datasets.

Embeddings vs. Related Terms#

• Latent space: The vector space that embeddings occupy. "Latent representation" and "embedding" are often used as synonyms.
• Feature extraction: Any process that turns raw data into features. An embedding is a feature vector learned so that distances between vectors carry meaning.
• Tokenization: Splits text into tokens and maps each to an integer ID. The embedding layer then maps each ID to a vector.
• One-hot encoding: Represents item i of a vocabulary as a vector of zeros with a 1 at position i, so every pair of items is equally far apart and the encoding carries no similarity. Multiplying a one-hot vector by an embedding table's weight matrix selects exactly one row, which is the result an embedding lookup returns without performing the multiplication.