Your eval suite has reported somewhere around 94% for five months. It went up slightly after a prompt change in June. Support ticket volume has not moved, the sentiment in those tickets has not improved, and nobody has connected the two observations.
The score is not wrong exactly. It is measuring something — just not the thing you think, and not the thing that changed. Evaluation suites decay in meaning while continuing to produce confident numbers, which makes this failure considerably more dangerous than a suite that simply broke.
What changed in 2026
- Continuous evaluation exposed the problem. Once teams ran evals constantly rather than before releases, the stability of the numbers became conspicuous — and stability across a period of real product change is a symptom.
- Judge validation became a recurring task. Model-based scoring is now common enough that judge reliability decaying over time turned into an operational concern rather than a research footnote.
- Contamination assumptions hardened. Public benchmark scores stopped being taken at face value, and a suspiciously strong result on a well-known set is now read as a warning.
- Multi-dimensional scorecards helped and did not solve it. Tracking cost and latency alongside accuracy caught some drift; it did not address a test set that stopped resembling production.
The four causes
They are independent, they present similarly, and the fix differs for each.
| Cause |
What moved |
Symptom |
| Product drift |
Your definition of correct |
Expected outputs are now wrong |
| Traffic drift |
What users actually send |
Test set no longer resembles production |
| Model drift |
The model under test |
Same prompt, different behaviour |
| Judge drift |
Your scoring model |
Scores shift with no system change |
Product drift is the most common and the easiest to miss. You changed a policy, added a feature, revised what a good answer looks like — and the test set still encodes last quarter's specification. Cases now failing correctly look like regressions; cases now passing may be passing for outdated reasons.
Traffic drift is the quietest. You built the set from what users sent a year ago. They send different things now — new phrasings, new languages, a feature that changed how people ask. The score is accurate about a population you no longer serve.
Model drift arrives with a version change and does not announce itself. Formatting compliance and refusal behaviour shift most readily, and neither shows up in an accuracy number.
Judge drift is the one that corrupts everything downstream. If a model scores your outputs, its agreement with human judgement can move — because the judge model changed, or because your outputs changed in ways the judge handles differently. Every metric built on it inherits the error.
Detecting it
The single most useful signal is divergence between your scores and any independent measure of quality. Scores flat while complaints rise. Scores rising while user behaviour is unchanged. When the eval and the world disagree, the eval is usually wrong.
Beyond watching for that, three habits catch drift early:
Re-label a sample by hand, quarterly. Take twenty cases your suite scores, have a human judge them fresh, and compare. Disagreement quantifies drift directly, and it is the only method that catches all four causes at once.
Keep a frozen holdout you never tune against. If holdout performance diverges from your main set, you have been fitting the test rather than improving the system. That divergence is the clearest evidence available, and you only get it if the holdout was genuinely never used.
Sample production into your test set continuously. New cases drawn from live traffic keep the set tracking reality. Online evals for LLM apps covers the sampling side, and it is the same pipeline that keeps a golden dataset current.
For judge drift specifically, run the judge against human-labelled cases periodically and track agreement as its own metric. A judge whose agreement fell from 0.9 to 0.7 has quietly invalidated months of reporting — see LLM-as-judge.
Fixing it without starting over
The instinct on discovering drift is to rebuild the suite. Usually unnecessary and often harmful, since you lose the history that made comparison possible.
Diagnose which cause moved. Product drift means updating expected outputs — and recording why, so the next maintainer can tell a spec change from a regression. Traffic drift means adding cases from current production. Model drift means re-validating the prompt against the new model. Judge drift means recalibrating or replacing the judge.
Only rebuild when the product changed so fundamentally that the old cases test a system that no longer exists. That happens, and it is rarer than the urge to declare it.
Common mistakes
- Trusting a stable score. Stability across real product change is a symptom, not reassurance.
- Never re-validating the judge. Its drift silently corrupts everything built on it.
- No frozen holdout. Removes your best evidence of test-set fitting.
- Updating expected outputs without recording why. The next person cannot distinguish a fixed spec from a hidden regression.
- Treating a public benchmark as a health check. Contaminated, general, and unrelated to your task.
- Rebuilding the suite on every drift. Destroys the historical comparison that makes scores useful.
FAQ
How often should I audit the suite?
Quarterly for a hand-labelled sample, and immediately after any model change or significant product change. More often is rarely worth the labour; less often and drift accumulates past the point of easy diagnosis.
Can I automate drift detection?
Partially. Judge-agreement tracking and holdout divergence automate well. Product drift fundamentally requires a human deciding what correct now means, which is not a thing to automate.
What if my scores go down after fixing drift?
That is the expected outcome and it is good news. The lower number is accurate; the higher one was measuring a stale specification. Communicate it as a correction rather than a regression, before someone else frames it as one.
Does this affect offline evals only?
Online evals drift too, mostly through judge drift and through the sampling strategy skewing as traffic patterns change. They are more resistant to traffic drift because they run on live traffic by construction.
Where to go next
For building the sets that drift in the first place, read golden datasets for LLM evals. For the production measurement that keeps them honest, online evals for LLM apps, and for the judge whose reliability underpins all of it, LLM-as-judge.