Faithfulness is the question of whether an answer says only what its sources support. It is the most tractable of the quality metrics because it is a comparison rather than a judgment — every claim either appears in the provided context or does not, and that has a determinable answer.
That tractability is why it has become the standard first metric for retrieval-augmented systems, and why its limits need stating clearly.
What changed in 2026
- Claim decomposition became standard. Splitting an answer into atomic statements before scoring each one replaced whole-answer scoring, which was too coarse to be actionable.
- Small judge models proved sufficient. Because entailment checking is a narrow task, compact models matched larger ones closely at much lower cost.
- Calibration reporting became expected. Publishing agreement between the judge and human labels moved from good practice to a standard requirement for trusting a score.
- The correctness gap got emphasized. Wider recognition that faithfulness certifies grounding rather than truth prevented over-reliance on it.
How to compute it
| Step |
What happens |
| 1. Generate the answer |
With retrieved context attached |
| 2. Decompose into atomic claims |
Each a single verifiable statement |
| 3. Classify each claim |
Supported, contradicted, or not addressed by the context |
| 4. Aggregate |
Proportion of claims supported |
| 5. Calibrate |
Compare judge labels against human labels on a sample |
Decomposition is the step that makes the metric useful. A paragraph containing four correct claims and one fabricated one scores poorly as a unit, which tells you something is wrong but not what. Decomposed, you get a precise proportion and, more importantly, the identity of the unsupported claim — which is what you actually act on.
The three-way classification matters too. A claim contradicted by the context is a different failure from one the context simply does not address. The first suggests the model misread the source; the second suggests it filled a gap from its own parameters. Both are ungrounded and they call for different fixes.
Faithfulness is not correctness
This distinction produces real confusion. Faithfulness asks whether the answer matches the source. Correctness asks whether the answer is true.
An answer perfectly summarizing an outdated policy document is fully faithful and entirely wrong for a user asking what the policy is now. A system scoring highly on faithfulness while retrieving stale documents is producing well-grounded misinformation — which is why the time-awareness work in temporal RAG explained sits upstream of this metric.
Measure both. Faithfulness diagnoses the generation stage; correctness diagnoses the whole pipeline including the corpus. A system that is faithful and incorrect has a retrieval or content problem, not a generation problem, and knowing which is the entire point of measuring separately — the argument in RAG evaluation metrics.
Common mistakes
- Scoring whole answers. Too coarse to act on.
- Binary supported-or-not classification. Loses the contradiction versus omission distinction.
- Uncalibrated judge scores. A number with no known error rate is not a measurement.
- Faithfulness as the sole gate. Certifies grounded answers from bad sources.
- Counting connective sentences as claims. Transitions and restatements are not factual assertions; excluding them avoids noise.
- Using an expensive judge. Entailment checking is a task small models handle well.
FAQ
What faithfulness score should I target?
High, and the useful comparison is against your own baseline over time rather than an absolute. Track the trend and investigate drops.
Can I measure faithfulness without retrieval?
Not meaningfully. Without provided context there is nothing to be faithful to; that becomes a correctness question against world knowledge.
How do I decompose claims automatically?
A model call splitting the answer into atomic statements works well, and it is worth spot-checking the decomposition itself since bad splits distort the score.
Does a high score mean no hallucination?
It means no ungrounded claims relative to the provided context. Hallucination in the broader sense includes retrieving the wrong context in the first place.
Where to go next
For attaching sources to claims, read citation grounding in LLMs. For the full measurement picture, RAG evaluation metrics, and for judge methodology, LLM as judge explained.