What is Reranking?
Also called Reranker, Cross Encoder Reranking.
Reranking is a second scoring pass that reorders an initial list of retrieved candidates using a more accurate and more expensive model. A fast retriever fetches perhaps fifty candidates, and the reranker examines each one against the query to produce a better ordering, from which only the top few are kept for reading.
Retrieval and reranking split the work by cost. An embedding based retriever encodes documents once in advance, so search compares precomputed vectors and stays fast at any corpus size. A cross encoder reranker instead reads the query and one document together, which is far more accurate and far too slow to run over an entire collection.
Running the cheap stage first and the expensive stage on its output captures much of the accuracy at a small fraction of the compute. Typical configurations retrieve between twenty and one hundred candidates and keep three to ten after reranking. The gain concentrates at the top of the list, which is exactly the part a generation step reads.
Rerankers come as dedicated cross encoder models, as hosted reranking endpoints, or as a general language model prompted to score relevance. The tradeoffs are latency, cost, and control. Because a reranker adds a round trip, latency sensitive applications sometimes rerank only when the retriever's top scores sit close together.
Reranking also improves precision in hybrid setups, where results arrive from two retrievers with incomparable scores. Scoring every merged candidate on one consistent scale resolves the ordering problem. Reranking cannot recover a correct passage the retriever never returned, so recall at the first stage remains the binding constraint on the whole pipeline.
Key points
- Second pass reorders a shortlist with a stronger model.
- Cross encoders read query and document together for accuracy.
- Adds latency and cost, so it runs on candidates only.
- Cannot recover documents the first stage never retrieved.
In practice
A query returns fifty candidate passages ranked by vector distance, with the truly correct passage sitting at position nineteen. A cross encoder scores all fifty against the query and lifts that passage to position one. The generation step now reads five passages, of which the first genuinely answers the question, and the answer improves with no change to the index.