Measuring an LLM is harder than measuring a classifier, because there is rarely one correct output. A summary can be good in ten different ways; an answer can be technically right and useless. That is why teams that ship reliable AI features spend as much effort on evaluation design as on prompting or model choice. A leaderboard score tells you almost nothing about whether your feature works for your users.
What changed in 2026
- LLM-as-judge became standard practice. Using a strong model to grade outputs at scale is now common, along with a growing awareness of its biases and the need to calibrate it.
- Benchmark contamination is taken seriously. Because public test sets leak into training data, teams lean harder on private, task-specific evals that models could not have memorized.
- Cost and latency joined the scorecard. With so many capable models available, evaluations increasingly weigh quality against price and speed rather than reporting accuracy in isolation.
Match the metric to the task
There is no universal LLM metric. What you measure depends on what the model is supposed to do.
| Task type |
Useful metrics |
Notes |
| Classification / extraction |
Accuracy, F1, exact match |
Deterministic; easy to automate |
| Summarization |
ROUGE, LLM-as-judge, human review |
Overlap metrics miss meaning; pair with a judge |
| Code generation |
pass@k, unit tests |
Execute the code; do not eyeball it |
| Open-ended chat |
LLM-as-judge, pairwise preference |
Rank against a baseline rather than scoring absolutely |
| RAG answers |
Faithfulness, answer relevance, context recall |
Separate retrieval quality from generation quality |
Overlap metrics like ROUGE and BLEU are cheap but weak — they reward surface word matches and miss paraphrase and meaning. Use them as a rough signal, not a verdict.
LLM-as-judge, used carefully
Letting a strong model grade outputs is the only way to evaluate open-ended tasks at scale. It works, but it drifts:
- Position bias: judges often favor whichever answer is shown first.
- Length bias: longer answers are frequently scored higher regardless of quality.
- Self-preference: a model may rate its own family of outputs more favorably.
Mitigate by randomizing order, capping length, using pairwise comparison instead of absolute scoring, and — most importantly — calibrating the judge against a few hundred human-labeled examples so you know its agreement rate. For RAG systems especially, judges pair well with retrieval metrics; see reranking in RAG for how retrieval quality feeds answer quality.
Building an eval set that means something
- Sample from real traffic. Draw inputs from actual or realistic user requests, including the messy and adversarial ones.
- Cover the edges. Include the failure cases you already know about so regressions are caught.
- Keep it private. A held-out set the model has never seen is the only defense against contamination.
- Version it. Freeze the eval set so scores are comparable across model and prompt changes.
- Measure the whole system. Evaluate the full pipeline — retrieval, prompt, and model together — not just the model in isolation.
FAQ
Are public benchmarks useless?
Not useless, but limited. They are good for coarse model triage and comparing broad capability, but contamination and generic tasks mean they rarely predict how a model performs on your specific workload.
What is pass@k?
For code generation, pass@k measures the probability that at least one of k sampled solutions passes the tests. It captures the fact that a model may succeed given a few attempts, which matches how developers actually use it.
How many examples does a good eval set need?
Enough to distinguish real differences from noise — often a few hundred well-chosen cases. Quality and coverage of edge cases matter more than raw count.
Can I trust LLM-as-judge scores without human review?
Only after calibration. Measure the judge's agreement with human labels on a sample first. If agreement is high on your task, you can lean on it; if not, keep humans in the loop for the decisive calls.
Where to go next