Question Answering

Question answering (QA) is the natural language processing (NLP) task of answering a question posed in natural language with the answer itself, such as "Canberra" for "What is the capital of Australia?" instead of a list of web pages. A QA system interprets the question, finds the information that answers it in a passage, a document collection, a database, or its own parameters, and returns or writes the answer. Search engine answer boxes, virtual assistants, customer support bots, and large language model (LLM) assistants that answer questions about company documents are all QA systems.

Early systems answered questions about narrow databases: BASEBALL (1961) handled questions about one season of American League games, and LUNAR (1971) answered geologists' questions about the Apollo moon rock samples. The TREC question answering track began benchmarking open-domain QA in 1999, and in February 2011 IBM Watson beat Jeopardy! champions Ken Jennings and Brad Rutter by generating many candidate answers to each clue and scoring the evidence for each with hundreds of algorithms.

Types of QA Systems#

• Extractive QA: Selects the span of a given passage that answers the question.
• Open-domain QA: Searches a large collection, such as all of Wikipedia, for relevant passages before answering.
• Retrieval-augmented generation (RAG): Writes the answer with an LLM conditioned on retrieved passages, so the answer can combine facts from several sources and cite them.
• Closed-book QA: Answers from knowledge stored in the model's weights, with no documents, which makes invented answers harder to detect.
• Knowledge graph QA: Translates the question into a structured query over a graph of entities and relations, or over a database.
• Visual question answering (VQA): Answers questions about an image or video.

Extractive QA and SQuAD#

The Stanford Question Answering Dataset (SQuAD), released in 2016 by Pranav Rajpurkar and colleagues, made extractive QA a standard benchmark: crowdworkers wrote more than 100,000 questions about Wikipedia articles, and every answer is a span of the article. A logistic regression baseline reached 51.0% F1, against 86.8% for humans. Scores use exact match and a token-level F1 score between the predicted and reference answers, and BERT reached a test F1 of 93.2 in 2018. An extractive model scores every passage token as a possible start and end of the answer:

import torch
from transformers import AutoModelForQuestionAnswering, AutoTokenizer

# DistilBERT fine-tuned on SQuAD v1.1 for extractive question answering
model_id = "distilbert/distilbert-base-cased-distilled-squad"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForQuestionAnswering.from_pretrained(model_id)

context = (
    "SQuAD was released in 2016 by Pranav Rajpurkar and colleagues at Stanford University. "
    "Crowdworkers wrote more than 100,000 questions about Wikipedia articles, and every answer is a span of the article."
)
for question in ["Who released SQuAD?", "Where do the answers come from?", "How many languages does SQuAD cover?"]:
    inputs = tokenizer(question, context, return_tensors="pt", return_offsets_mapping=True)
    offsets = inputs.pop("offset_mapping")[0]
    with torch.no_grad():
        outputs = model(**inputs)

    # The answer is the context span between the highest-scoring start and end tokens
    start, end = outputs.start_logits.argmax(), outputs.end_logits.argmax()
    print(f"{question} -> {context[offsets[start][0] : offsets[end][1]]}")

The model answers "Pranav Rajpurkar" and "a span of the article." The passage says nothing about languages, yet the model answers "100,000" to the third question, because a model trained on SQuAD 1.1 always returns some span. SQuAD 2.0 (2018) added more than 50,000 unanswerable questions written to look answerable, and a system that scored 86% F1 on SQuAD 1.1 dropped to 66% on it, since models now had to decide when to abstain.

Open-Domain QA and RAG#

Open-domain systems split the work between a retriever, which finds a few relevant passages among millions, and a reader, which answers from them. DrQA (2017) retrieved Wikipedia articles with TF-IDF and read them with a recurrent neural network, Dense Passage Retrieval (2020) replaced TF-IDF with BERT embeddings and vector search, and RAG (2020) handed the retrieved passages to a generator that writes the answer. A reranker often reorders the retrieved passages before the reader sees them. The RAG authors found that conditioning on retrieved text made generated language more specific, diverse, and factual than a comparable model without retrieval, and cited passages let readers check an answer for hallucination.

Visual Question Answering#

VQA takes an image and a question such as "How many people are waiting at the stop?" and requires multimodal AI that combines computer vision with language understanding, usually a vision-language model (VLM). In Ultralytics Platform Agents, an LLM block sends an image and its upstream context, such as YOLO26 detections, to a vision-language model along with the prompt, so a question about a scene is answered with the detector's results as context.

QA vs. Related Terms#

• Semantic search: Returns the documents or passages most relevant to a query, the input a QA system reads to produce the answer.
• Chatbots: Conversational interfaces that manage multi-turn dialog, and they use QA for factual questions within it.
• Text generation: Produces any text from a prompt. Generative QA is text generation constrained to answer a question, often from cited sources.