Evals are the least glamorous and most important part of building AI products. Without them, every prompt change is a guess, every model upgrade is a prayer, and "it worked in my testing" is the closest thing to confidence you get. In 2026, as AI features move deeper into critical workflows, "we eyeballed it" is no longer an acceptable quality bar.
What changed in 2026
- LLM-as-judge became standard. Using a strong model (Claude Opus, GPT-4o) to grade outputs from a weaker model is now the default approach for quality evals — it scales and correlates well with human preference when the rubric is tight.
- Eval frameworks matured. Tools like Braintrust, LangSmith, and Weights & Biases Weave provide first-class eval pipelines with dataset versioning, human annotation interfaces, and regression dashboards.
- Golden dataset curation is the hard part. Everyone has tools; the differentiator is a well-labeled eval set that covers failure modes specific to your use case.
- Synthetic data generation fills gaps. Generating adversarial and edge-case examples with an LLM is now standard practice for bootstrapping an eval set.
Types of evals and when to use each
| Eval type |
What it measures |
Best for |
| Exact match |
Output matches a reference string |
Classification, structured extraction |
| Rubric-based (LLM-judge) |
Quality on defined criteria |
Open-ended generation, tone, helpfulness |
| Retrieval hit rate |
Did the right chunks get retrieved? |
RAG pipelines |
| Faithfulness |
Is the answer grounded in the retrieved context? |
RAG, summarization |
| Tool call accuracy |
Did the model call the right tool with right args? |
Agents |
| Latency / cost per task |
Efficiency |
Production monitoring |
| Human preference |
Which output do humans prefer? |
Final quality bar |
The LLM-as-judge pattern
For open-ended tasks, write a rubric and ask a grader model to score on it:
You are evaluating an AI assistant's answer. Score it 1–5 on each criterion:
- Accuracy: Is the answer factually correct?
- Completeness: Does it address all parts of the question?
- Conciseness: Is it appropriately brief?
Question: {question}
Reference answer: {reference}
Model answer: {model_output}
Output a JSON with keys: accuracy, completeness, conciseness, reasoning.
Key practices: use a temperature of 0, require structured output (JSON), and calibrate the rubric against a sample of human-labeled examples before trusting the scores.
Building your eval dataset
- Start with production logs. Real user queries are better than synthetic ones.
- Sample diverse failure modes. Don't just test the easy cases — pull queries the model got wrong during QA.
- Add adversarial examples. Prompt injections, ambiguous queries, edge cases from your domain.
- Label conservatively. Prefer smaller, high-quality datasets over large noisy ones.
- Version and freeze. Your eval set is a dataset artifact — track it like code.
Target: 100–500 examples to start; 1,000+ for production-grade regressions. More is not always better if labels are inconsistent.
How to pick what to evaluate first
- What would users complain about first? Start there.
- What's the highest-stakes output in your pipeline? That needs the most rigorous eval.
- What changed last sprint? Always run evals on the component that was modified.
- Where did the last production incident come from? Add a test for that exact failure.
Common mistakes
Leaking the eval set into training or prompting. If your few-shot examples come from your eval set, your scores are inflated. Keep them strictly separate.
Optimizing for the eval, not the goal. If your rubric doesn't capture what users actually care about, you'll improve your score while degrading real quality.
Running evals only before launches. Evals should run on every significant prompt or model change — treat them like CI tests.
Using a weak judge model. If you're grading GPT-4o outputs with GPT-3.5, the judge is less capable than the system under test. Use the strongest model you can afford for grading.
Ignoring cost and latency metrics. Quality evals without efficiency metrics lead to gold-plated models that nobody can afford to run.
What to skip
- Academic benchmarks as a proxy for your use case — MMLU and HumanEval tell you nothing about your customer support chatbot.
- Manual grading at scale — human annotation is the ground truth for calibration, not the production eval signal; it doesn't scale.
- Aggregate scores without distribution — a mean score hides tail failures; always look at p10 and failure rate too.
FAQ
How many examples do I need for a reliable eval?
100 examples gives statistically meaningful signal for detecting a 5-point quality change. For detecting small regressions, 500+ is more reliable.
Can I use the same model to evaluate itself?
No — it introduces self-preference bias. Use a different, preferably stronger, model as the judge.
How do I handle subjective tasks?
Acknowledge subjectivity in the rubric, use multiple judge calls and average them, and calibrate against human labels on a representative sample.
What's a reasonable eval cadence?
Run fast evals (LLM-judge on 100 examples) in CI on every PR touching prompts. Run comprehensive evals (full dataset, human review) on model version changes.
Where to go next