What is Mean Reciprocal Rank?
Also called MRR.
Mean reciprocal rank is a retrieval metric that scores each query by one divided by the position of its first relevant result, then averages across all queries. A first-place hit scores 1.0, second place 0.5, third 0.33. It rewards putting a correct result at the very top and ignores everything after the first hit.
The reciprocal weighting encodes a specific assumption: users read from the top and stop at the first useful item. Moving a relevant document from rank three to rank one raises its contribution from 0.33 to 1.0, while moving it from rank nine to rank seven changes almost nothing. That steep curve makes the metric sensitive precisely where user experience is most sensitive.
Its blind spot is everything beyond the first relevant hit. A query with five relevant documents scores identically whether the system found one of them or all five, as long as the top-ranked one is in the same position. For questions that require synthesizing several sources, which is common in retrieval augmented generation, this makes mean reciprocal rank an incomplete picture on its own.
It also depends on the cutoff used. If no relevant document appears within the evaluated depth, that query contributes zero, so the metric mixes together failure to retrieve and failure to rank. Reporting hit rate at the same cutoff alongside it separates those two, since hit rate says whether the document was found and mean reciprocal rank says how well it was placed.
The metric is best suited to tasks with one obviously correct answer, such as looking up a specific policy, a definition, or a known document. For tasks needing broad coverage, normalized discounted cumulative gain or recall at a cutoff describe the outcome better. Most teams track two or three metrics together rather than optimizing any single one.
Key points
- Scores one over the rank of the first relevant result
- Steeply rewards top-position hits
- Ignores additional relevant documents beyond the first
- Zero when nothing relevant appears within the cutoff
- Best for single-answer lookups, weak for coverage tasks
In practice
Across four test queries, the first relevant passage lands at ranks one, three, two, and nowhere in the top ten. The reciprocal ranks are 1.0, 0.33, 0.5, and 0, giving a mean reciprocal rank of about 0.46. Hit rate at ten is 0.75. Read together, the pair shows that retrieval usually finds the passage but often ranks it below the top slot.