What is Retrieval Pipeline?
Also called Retrieval Stack, Search Pipeline.
A retrieval pipeline is the ordered sequence of steps that turns a user question into the passages a model reads, typically query processing, retrieval, merging, filtering, reranking, and context assembly. Treating it as a pipeline makes each stage measurable and replaceable, which is how retrieval quality is diagnosed and improved.
The first stage transforms the query. Common operations include rewriting a follow up question into a standalone one using conversation history, expanding it with synonyms or alternate phrasings, and generating several sub queries for compound questions. Skipping this stage is a frequent cause of poor retrieval on conversational input.
Retrieval follows, often running lexical and vector search in parallel and fusing the results. Filters for tenant, permission, date, or document type are applied here so that ineligible material never reaches later stages. A reranker then scores the merged shortlist, and a selection step keeps as many passages as the reading budget allows.
Context assembly is the final stage and is easy to underrate. Passages are deduplicated, ordered, labeled with source and date, and possibly expanded to include surrounding text. The assembled block, together with instructions on how to use it and when to abstain, is what actually determines the answer the model produces.
Because each stage can fail independently, instrumentation is essential. Logging the rewritten query, the candidate identifiers and scores at each stage, and the final assembled context makes failures diagnosable, showing whether a passage was never retrieved, retrieved and dropped by a filter, or retrieved and then ignored by the model.
Key points
- Ordered stages from query rewriting to context assembly.
- Filters belong early, before reranking and reading.
- Each stage is separately measurable and replaceable.
- Log candidates per stage to locate failures.
- Assembly and instructions shape the final answer.
In practice
A user asks, and what about the annual one. The pipeline rewrites this to what is the refund window for the annual plan using prior turns, runs hybrid retrieval scoped to the user's workspace, fuses forty candidates, reranks to five, and assembles them with titles and dates. Logs show the winning passage entered at rank twelve.