Agentic RAG

Agentic RAG is retrieval-augmented generation (RAG) in which an AI agent, a large language model (LLM) that plans and calls tools in a loop, decides whether to retrieve, which sources to search, and whether the results are good enough to answer from. Standard RAG runs one fixed pass: embed the question, fetch the top passages, generate. An agent can split a question into sub-questions, send one to a document index and another to a SQL database or web search, notice that a result is off-topic, rewrite the query, and search again. A 2025 survey of agentic RAG groups these behaviors into four agentic design patterns: reflection, planning, tool use, and multi-agent collaboration.

How Agentic RAG Works#

The model reaches its retrieval tools through function calling. Each tool, such as a vector search over documents, a database query, or a web search, is described by a name and a parameter schema, and the model emits a structured call whenever it wants to use one. The Model Context Protocol (MCP) gives agents a standard way to connect to such tools. The control loop follows the ReAct pattern of alternating reasoning steps and actions:

A question such as "Which suppliers had more late deliveries this quarter than last, and what penalties do their contracts allow?" shows why the loop matters. A single retrieval pass embeds the whole question and returns contract passages that mention late deliveries in general. An agent first queries the delivery database for counts per supplier, then searches the contract index once for each supplier whose count rose, and combines both sources in its answer.

Tools are not limited to text. When an answer depends on what a camera sees right now, an agent can call an Ultralytics YOLO26 model as a tool: prediction on the latest frame returns every detected object, and results[0].summary() turns the detections into a list of dictionaries with class name, confidence, and box coordinates that the agent reads like any other retrieved result.

Agentic RAG vs. Related Approaches#

• Standard RAG: Retrieves once per question through a fixed pipeline, with low and predictable latency and cost. It suits questions that one or two passages can answer.
• Self-RAG and Corrective RAG: Self-RAG (2023) trains a model to emit reflection tokens that decide when to retrieve and grade its own output, and Corrective RAG (2024) adds a retrieval evaluator that triggers a web search when the retrieved documents score poorly. Both place one piece of agent behavior inside an otherwise fixed pipeline.
• GraphRAG: Retrieves entities, relationships, and community summaries from a knowledge graph built from the documents. An agent can call graph queries as one of its tools.
• Agentic workflows: The broader family of multi-step LLM systems, of which agentic RAG is the member focused on finding information.
• Deep research agents: Agentic RAG stretched over many searches for one question, often split across several agents. In Anthropic's multi-agent research system, a Claude Opus 4 lead agent hands subtasks to parallel Claude Sonnet 4 subagents, and the system outperformed a single Opus 4 agent by 90.2% on Anthropic's internal research evaluation.

Real-World Applications#

• Enterprise research assistants: Answer questions that span a data warehouse, document stores, and the web, such as comparing internal sales figures with a competitor's published results.
• Customer support: An agent checks an order's status in one system, retrieves the returns policy from another, and decides whether a refund applies before replying.
• Manufacturing troubleshooting: An assistant retrieves the maintenance log and manual pages for the machine named in a fault report, and searches past incident tickets when the manual does not cover the error code.
• Agentic coding: Coding agents search a repository, read the files they find, and search again with names discovered along the way before editing code.

Costs and Failure Modes#

Every extra step adds an LLM call, so agentic RAG is slower and more expensive than a single pass. Anthropic's analysis of its multi-agent research system found that agents typically use about 4 times as many tokens as chat interactions, and multi-agent systems about 15 times as many. Agents also fail in ways fixed pipelines do not: they loop on similar queries, stop before finding the key document, call the wrong tool, or carry an early mistake through every later step, which can surface as confident hallucinations. Production systems cap the number of steps, log full traces, and score whole trajectories with agent evaluation. In Anthropic's analysis, token usage alone explained 80% of the variance in performance on the BrowseComp browsing benchmark.