What is Query Rewriting?
Also called query reformulation.
Query rewriting transforms a user's raw input into a better search query before retrieval runs. Typical rewrites resolve pronouns using conversation history, make an implied subject explicit, strip conversational filler, or split a compound question into parts. It exists because what a person types is often a poor query even when their intent is clear.
The dominant use is conversational reference resolution. A follow-up such as what about the second one carries almost no retrievable content on its own; against a corpus it matches nothing useful. A rewriter reads the recent turns and produces a standalone query naming the actual entity, which is why this step is sometimes called query decontextualization or history-aware retrieval.
Rewriting also normalizes shape. Chat input often mixes a request, an aside, and a constraint in one message. A rewriter can drop the politeness and the aside, keep the constraint as a metadata filter, and emit a short focused query. Compound questions that ask two things at once can be split, retrieved for separately, and their results merged before an answer is drafted.
The main risk is intent drift. A rewriter that paraphrases too freely can substitute a related but different question, and the failure is hard to spot because retrieval then works well for the wrong query. Practical safeguards include keeping the original query in the retrieval mix, logging every rewrite for inspection, and constraining the rewriter to minimal edits rather than full reinterpretation.
Cost matters too, since a rewrite adds a model call in front of every search and therefore adds latency to the most user-visible part of the pipeline. Many systems apply rewriting selectively, triggering it only for short follow-up turns or when the first retrieval returns weak similarity scores, which keeps the average request fast while still rescuing the hard cases.
Key points
- Turns context-dependent input into a standalone query
- Resolves pronouns and implied subjects from chat history
- Can split compound questions into separate retrievals
- Free paraphrasing risks silently changing user intent
- Often applied selectively to control added latency
In practice
A user asks about the refund window for annual plans, then follows up with and for monthly. Sent verbatim, that follow-up retrieves nothing relevant. The rewriter emits refund window for monthly plans, which matches the billing policy passage directly. The system logs both the original and rewritten text, so a reviewer can later confirm the rewrite preserved what the user actually meant.