What is Faithfulness?
Also called groundedness.
Faithfulness measures whether every claim in a generated answer is supported by the retrieved passages given to the model. It asks only about consistency with the supplied evidence, not about real-world truth, so an answer faithfully repeating an incorrect source still scores well. It is the standard way to detect unsupported additions in retrieval augmented systems.
Evaluation typically decomposes the answer into individual factual statements, then checks each one against the retrieved context. The score is the fraction of statements that the context supports. Automating this usually means a second model acting as a judge, with natural language inference models as a lighter alternative, and human spot checks to confirm the judge is not itself hallucinating support.
The narrow definition is deliberate. Separating consistency with sources from correctness of sources means a low faithfulness score points at generation behavior, while a wrong but faithful answer points at the corpus or the retriever. Collapsing the two into one accuracy number destroys that diagnostic value and leaves teams unable to tell which component to fix.
Common causes of low faithfulness include a model filling gaps from its own parameters when the context is thin, subtle overreach such as turning may into will or generalizing a single example into a rule, and combining two passages into a conclusion neither states. Numeric claims and dates deserve extra scrutiny, since these are where small unsupported edits do the most damage.
Faithfulness must be read together with a relevance measure. An answer that says the documents do not cover this question is perfectly faithful and often useless. Tracking faithfulness alongside answer relevance and context precision gives a balanced view, and pairing them with citations lets a reader verify the connection between claim and source directly.
Key points
- Checks answer claims against the retrieved passages only
- Does not measure real-world truth of the sources
- Usually scored by decomposing the answer into claims
- Watch for overreach on numbers, dates, and modal verbs
- Meaningless without a relevance measure alongside it
In practice
Retrieved context states that the standard plan includes ten seats. The answer says the standard plan includes ten seats and additional seats can be purchased monthly. The first claim is supported; the second appears nowhere in the context. Faithfulness for this answer is 0.5, and the added sentence is exactly the kind of plausible unsupported detail that a reader would never think to question.