Vector Database

A vector database stores embeddings, the numeric vectors that models produce for text, images, audio, or video, together with an ID and metadata for each item, and answers the query "which stored vectors are closest to this one?" in milliseconds across millions or billions of items. It wraps a vector search index in the features of a database: inserts, updates, and deletes, filtering by metadata, persistence, access control, and replication across machines. Vector databases provide the retrieval step of retrieval-augmented generation (RAG), semantic search, and recommendation systems.

How a Vector Database Works#

On the write path, an embedding model, either outside the database or built into it, converts each item into a vector, and the database stores that vector with an ID and a metadata payload such as the source file, date, camera, or customer. Comparing a query with every stored vector is exact but slow at scale, so the database builds an approximate nearest neighbor (ANN) index. HNSW links each vector to its near neighbors in a layered graph that a search walks greedily; inverted file (IVF) indexes group the vectors with k-means clustering and scan only the clusters closest to the query; and vector quantization methods such as product quantization compress each vector to a few bytes. On the read path, the query is embedded with the same model and compared using a metric such as cosine similarity, dot product, or Euclidean distance, and the database returns the top-k IDs, scores, and payloads, which many systems merge with keyword results in hybrid search.

This example uses Chroma, an open-source vector database that runs in memory inside a Python process:

import chromadb

# In-memory client; chromadb.PersistentClient(path="db") keeps the data on disk
client = chromadb.Client()
collection = client.create_collection("inspection_notes")

# Chroma embeds each document with its default embedding model and indexes the vectors
collection.add(
    ids=["n1", "n2", "n3"],
    documents=[
        "Hairline crack found near the weld on bracket B7",
        "Paint scratch on the left door panel",
        "Porosity in the weld seam of the frame rail",
    ],
    metadatas=[{"line": "A"}, {"line": "B"}, {"line": "B"}],
)

# Nearest-neighbor search restricted to one production line
results = collection.query(query_texts=["welding defects"], n_results=1, where={"line": "B"})
print(results["ids"][0], results["documents"][0])

The query returns note n3, the weld porosity on line B. The metadata filter removes the line A crack note from consideration, although its text is just as close to "welding defects."

Vector Databases vs. Related Systems#

• Vector search libraries such as FAISS: In-process indexes, including HNSW, IVF, and product quantization, with no server, metadata storage, or replication. Several vector databases build on them.
• Purpose-built vector databases: Pinecone as a managed service, and Milvus, Qdrant, Weaviate, and Chroma as open-source projects, designed around embeddings from the start.
• Vector extensions such as pgvector: Add vector columns with HNSW and IVFFlat indexes to PostgreSQL, so embeddings live next to relational data. Elasticsearch, OpenSearch, MongoDB Atlas, and Redis offer similar vector fields next to their keyword search.
• Feature stores: Serve model input features by entity key for training and inference. They look up records by ID, not by similarity.
• Data lakes: Hold raw files such as images and logs, which a vector database references by ID or path.

Real-World Applications#

• Memory for AI agents: RAG pipelines and agents store document chunks and past conversations so a large language model can retrieve them on demand.
• Retail visual search: "Shop similar" features embed every product photo and return the nearest items to the one a shopper is viewing.
• Anomaly detection: A new sample whose nearest stored neighbors are all far away is flagged as unusual, for example a transaction or a sensor reading.

Image collections need a vision model to produce the vectors. Ultralytics YOLO26 provides one through model.embed(), which runs the model on each image and returns a feature vector pooled from its second-to-last layer, ready to store next to the image path and capture metadata for near-duplicate detection or similar-image search, the kind of lookup behind Find Similar Images in Ultralytics Platform.

Sizing and Tuning#

Index settings trade recall against inference latency: HNSW's search breadth (ef_search) and IVF's number of probed clusters (nprobe) both find more true neighbors at the cost of slower queries, so teams tune them against exact search on sample queries. Memory sets the hardware bill. One million 768-dimensional float32 embeddings take 3.07 GB (1,000,000 x 768 x 4 bytes) before any index overhead, int8 scalar quantization of the vectors cuts that to 768 MB, and Matryoshka embeddings can be truncated to fewer dimensions for a further reduction.