Recommendation System

A recommendation system, or recommender, is software that predicts which items a user is likely to want, such as products, videos, songs, articles, or posts, and ranks them for that user. It learns from explicit feedback such as star ratings and, far more often, from implicit feedback such as clicks, watch time, and purchases, which makes it one of the most widely deployed applications of machine learning. Two milestones shaped the field: Amazon's 2003 description of item-to-item collaborative filtering, and the Netflix Prize (2006–2009), which paid $1 million to the team that improved Netflix's movie-rating predictions by 10%.

How Recommendation Systems Work#

Collaborative filtering uses only the pattern of who interacted with what. User-based methods find users with similar histories, item-based methods find items liked by the same users, and matrix factorization, popularized during the Netflix Prize, learns a vector for every user and item whose dot product predicts their interaction. Content-based filtering instead describes items by their attributes, such as genre, text processed with natural language processing (NLP), or image embeddings, and recommends items similar to those a user already liked, which works even for items nobody has interacted with yet. Hybrid systems combine both signals so that content features can place a new item until enough interactions accumulate.

This example applies item-based collaborative filtering to a small ratings matrix, predicting one user's ratings of the movies they have not rated from the cosine similarity between movies' rating columns:

import numpy as np

# Ratings from 4 users (rows) for 5 movies (columns); 0 means not rated
ratings = np.array(
    [
        [5, 4, 0, 1, 0],
        [4, 5, 1, 0, 1],
        [1, 0, 5, 4, 5],
        [0, 1, 4, 5, 0],
    ]
)

# Item-item cosine similarity between the movies' rating columns
norms = np.linalg.norm(ratings, axis=0)
item_sim = ratings.T @ ratings / np.outer(norms, norms)

# Predict user 3's rating of each movie as a similarity-weighted average of their own ratings
user = ratings[3]
rated = user > 0
scores = item_sim[:, rated] @ user[rated] / item_sim[:, rated].sum(axis=1)

for movie in np.flatnonzero(~rated):
    print(f"Movie {movie}: predicted rating {scores[movie]:.2f}")

User 3, the last row, rated movies 2 and 3 highly and movie 1 poorly, so movie 4, whose ratings track movies 2 and 3, gets a predicted rating of 4.10, and movie 0, whose ratings track movie 1, gets 2.09. The system recommends movie 4.

Deep learning recommenders learn these representations end to end. Two-tower networks embed users and items separately, so retrieving candidates becomes a vector search for the item embeddings nearest to a user embedding, and sequence models such as transformers read a user's history in order to predict the next interaction.

The Multi-Stage Pipeline#

Large services cannot run a heavy model on millions of items for every request, so they recommend in stages. Candidate generation, or retrieval, narrows the catalog to a few hundred items with cheap signals such as collaborative filtering or embedding search. A ranking model then scores each candidate with rich features to predict the probability of a click, watch, or purchase, and a final re-ranking pass adds diversity, freshness, and business rules. YouTube described this two-stage design in 2016. Every recommendation shown also generates the feedback used to train the next model, so the system learns from data its own earlier choices shaped.

Recent systems scale the ranking and retrieval models like language models. Meta's generative recommenders, published at ICML 2024, treat each user's actions as a sequence, reached 1.5 trillion parameters, and improved metrics in online A/B tests by 12.4%.

Visual Recommendations#

Fashion, furniture, and art catalogs recommend by appearance, so "more like this" features compare image embeddings of product photos, and "shop the look" features first use object detection to crop each item out of a scene photo before searching the catalog for visually similar products. The same retrieval suggests training data: Find similar images in Ultralytics Platform shows up to 24 of the nearest images from public and team datasets, each with a similarity score, skips images the dataset already holds, and can start from a model's worst-performing validation images.

Applications#

• Retail and e-commerce: Product pages show "customers who bought this also bought" lists and personalized home pages built from browsing and purchase history.
• Streaming media: Video and music services rank titles and generate playlists from viewing and listening history, and Netflix also personalizes the artwork shown for each title.
• Social feeds and news: Platforms rank posts, short videos, and articles by predicted engagement, refreshing the ranking every time a user opens the feed.
• Fashion: Visual similarity search suggests garments that match the style, color, or pattern of an item a shopper viewed or photographed.

Recommendation Systems vs. Related Concepts#

• Search: A search engine answers an explicit query, while a recommender anticipates what a user wants without one; modern systems share their retrieval and ranking components.
• Vector search and vector databases: Find the stored embeddings nearest to a query embedding, the building block a recommender uses for retrieval.
• Rerankers: Second-stage models that rescore a short candidate list more precisely, the role of the ranking stage in a recommender.
• Predictive modeling: The ranking stage is a predictive model of interaction probability, while the full recommender also handles retrieval, ordering, and the feedback loop.

Challenges#

• Cold start: New users and new items have no interaction history, so systems fall back on popularity, onboarding questions, or content features.
• Sparsity: Each user interacts with a tiny fraction of a large catalog, so most of the user-item matrix is empty.
• Popularity bias and feedback loops: Training on logged interactions favors items that were already shown often, which narrows what users see and can create filter bubbles.
• Evaluation: Offline metrics such as precision@k and NDCG on historical data often disagree with online A/B tests, which measure what users actually do with new recommendations.
• Privacy: Detailed behavioral profiles are personal data, protected in the EU and other jurisdictions by data protection law.
• Regulation: The EU Digital Services Act requires very large online platforms and search engines to offer, for each recommender system, at least one option that is not based on profiling.