Reranker

A reranker is a model that takes a short list of candidates from a fast first-stage retriever, scores each candidate against the query more carefully, and reorders the list. Search engines, retrieval-augmented generation (RAG) pipelines, and recommendation systems use the pattern because the most accurate relevance models are too slow to run over millions of documents but fast enough for the top few dozen to few hundred. In text search the standard reranker is a cross-encoder, a transformer that reads the query and one candidate together and outputs a single relevance score. Nogueira and Cho (2019) fine-tuned BERT this way and topped the MS MARCO passage ranking leaderboard with an MRR@10 27% higher, in relative terms, than the previous best.

How Retrieve-then-Rerank Works#

The first stage favors recall: keyword matching with BM25, vector search over precomputed embeddings, or both, pulls a few hundred plausible candidates from the whole collection in milliseconds. The reranker favors precision: it spends a full model pass on every query-candidate pair, and only its top few results go to the user or into an LLM's context window. The Ultralytics VisualAISearch solution is a first-stage retriever of this kind, embedding a folder of images with CLIP ViT-B/32, ranking them by cosine similarity to a text query, and returning up to 30 matches above a 0.1 similarity threshold by default.

Cross-Encoders and Bi-Encoders#

A bi-encoder, the usual retriever, embeds the query and each document separately, so document vectors can be computed once and stored in a vector database. That independence is also its limit: the document vector is fixed before the query is known. A cross-encoder concatenates query and document into one input, so self-attention can compare every query word with every document word, at the cost of a full forward pass for every pair at query time. ColBERT sits between the two, storing one vector per document token and matching query tokens against them with late interaction. The Sentence Transformers CrossEncoder class runs a reranker such as ms-marco-MiniLM-L6-v2, trained on MS MARCO search queries:

from sentence_transformers import CrossEncoder

query = "When was the transformer architecture introduced?"
# Shortlist returned by a fast first-stage retriever, in its original order
candidates = [
    "Transformer models such as GPT and BERT are widely used in natural language processing.",
    "An electrical transformer changes the voltage of an alternating current.",
    "Convolutional neural networks became popular after AlexNet won ImageNet in 2012.",
    "The transformer was introduced in the 2017 paper Attention Is All You Need.",
]

# The cross-encoder reads the query and each candidate together and scores their relevance
reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")
ranked = reranker.rank(query, candidates)

for hit in ranked:
    print(f"{hit['score']:6.2f}  {candidates[hit['corpus_id']]}")

The passage that answers the question moves from last place to first with a score of 6.97, far ahead of the next candidate at -4.48. The electrical transformer sentence shares the query's key word but scores -5.29.

Types of Rerankers#

• Cross-encoders: Compact transformers fine-tuned on relevance labels, such as the MS MARCO MiniLM models and BAAI's bge-reranker family, also offered as hosted reranking APIs.
• LLM rerankers: A large language model (LLM) is prompted to score candidates or sort a whole list at once. RankGPT showed that ChatGPT and GPT-4 can rerank passages by writing out a sorted list, and distilled ChatGPT's rankings into a small dedicated reranker.
• Learning-to-rank models: Recommendation and e-commerce systems rerank candidates with models such as gradient-boosted trees over features like click history, price, and freshness, trained with ranking objectives such as LambdaMART, available in LightGBM and XGBoost.

Rerankers vs. Related Concepts#

• Semantic search: Search by meaning, usually with a bi-encoder retriever. A reranker is an optional second stage on top of it.
• Hybrid search: Merges keyword and vector result lists, often with reciprocal rank fusion, which combines ranks without reading the content again. A reranker often runs on the merged list.
• Knowledge distillation: Trains a fast retriever to imitate a slow reranker's scores, moving some reranking quality into the first stage.
• Two-stage object detectors: Follow the same pattern for images, with a region proposal stage suggesting candidate boxes and a second stage re-scoring and refining each one before non-maximum suppression (NMS) removes overlapping duplicates.