What is Agentic Retrieval?
Also called agentic RAG.
Agentic retrieval lets a model decide for itself whether to search, what to search for, and when it has enough evidence, instead of running one fixed retrieval before every answer. The model can issue several searches in sequence, refine queries based on what came back, and choose among different sources or tools.
A fixed pipeline retrieves once with whatever the user typed, which handles simple lookups well and multi-part questions poorly. An agentic setup exposes search as a tool the model may call repeatedly, so a question requiring a definition, a policy, and a recent figure can be answered through three targeted searches rather than one blurred query that partially matches all three.
The pattern also enables source selection. Given a lexical index, a vector index, a structured database, and a web search tool, the model can route an identifier lookup to the lexical index and a conceptual question to the vector index. Descriptions of each tool matter enormously here, since routing quality depends on the model correctly understanding what each source contains.
Self-assessment is the other half. The model can judge whether retrieved passages actually answer the question and search again with different wording if they do not, which recovers cases a single-shot pipeline would fail silently. This is closely related to reflection-style loops in agent design, and like all such loops it needs a hard iteration cap to avoid unbounded searching.
The costs are latency, spend, and reduced predictability. Each search is a round trip plus a model call, so response times become variable and harder to promise. Debugging is harder because two identical questions can take different paths, which makes logging the full search trajectory, not just the final answer, a practical requirement rather than a nicety.
Key points
- The model chooses when, what, and how often to search
- Supports multi-step questions and multi-source routing
- Can detect weak results and retry with new wording
- Needs a hard cap on search iterations
- Latency and cost become variable and harder to predict
In practice
Asked whether the current refund policy differs from last year's, the model first searches for the current policy, notices no comparison is present, then searches the archive for the prior version, then compares the two retrieved passages. A single-shot pipeline would have retrieved only current-policy passages and produced an answer that quietly ignored half the question.