Safety evals are structured tests that probe whether a model behaves acceptably under adversarial, ambiguous, or high-stakes conditions — not whether it is capable, but whether its capability is used the way it should be. A model can write flawless code and still fail a safety eval by leaking private data when asked cleverly enough, or by giving dangerous instructions when the request is phrased to slip past a filter. Capability and safety are measured separately because a model can excel at one while failing the other.
What changed in 2026
- Evals shifted from static question sets to interactive testing. Instead of a fixed list of prompts, current safety evals increasingly use multi-turn conversations and agentic tasks, since single-turn prompts undersell how a model behaves once a user has several turns to steer it.
- Third-party and government-affiliated eval bodies expanded. More safety testing now happens outside the lab that built the model, addressing the conflict of interest in a vendor grading its own homework.
- Tool-use safety became its own category. As models gained the ability to call external tools and take actions, evals expanded to test whether a model will misuse a tool, not just whether it will say something harmful.
- Eval gaming became a recognized risk. Once specific eval sets became well known, some models showed signs of performing better on the exact test than on close variants — pushing eval designers toward held-out and frequently rotated test sets.
What safety evals actually check
Safety evals generally cluster into a few categories: harmful content generation (does the model produce content that facilitates real-world harm), deception (does the model knowingly state falsehoods or hide its reasoning), unsafe tool use (does the model take a harmful action when given tool access), and robustness to adversarial prompts (does a jailbreak attempt succeed). Each category needs a different testing method — content evals can often be automated with a grading model, while tool-use and deception evals usually require carefully constructed scenarios and human review.
Automated evals vs red teaming
| Approach |
Strength |
Weakness |
| Automated eval suites |
Fast, cheap, repeatable, good for regression testing |
Limited to known failure patterns; easy to game if static |
| Human red teaming |
Finds novel, creative failure modes |
Slow, expensive, hard to scale across every model update |
| Model-graded evals |
Scales better than pure human review |
Grading model can share blind spots with the model being tested |
| Production monitoring |
Catches real-world misuse patterns |
Reactive by nature; the harm has already reached a user |
Most serious safety programs run all four, not as a substitute for one another but as layers that catch different failure classes. This is a similar pattern to how AI observability treats pre-launch testing and post-launch monitoring as two halves of the same problem.
Why passing evals is not a guarantee
A safety eval suite is bounded by what its designers thought to test. Real users are creative in ways a test suite is not, and a model that never encountered a particular attack pattern during eval can still be vulnerable to it once it meets the much larger and more inventive population of real users. This is why eval results should be read as "no known failures on these specific tests" rather than "safe."
Common pitfalls
- Testing the base model but not the deployed system. A safety eval on the raw model does not capture how a system prompt, retrieval layer, or tool integration changes its behavior — the eval needs to run against what actually ships.
- Treating one eval score as the whole picture. A single aggregate safety score hides which specific failure categories are weak; teams that only track the aggregate miss where the real risk sits.
- Skipping re-evaluation after fine-tuning. Fine-tuning for capability can quietly shift safety behavior. Any material change to the model or its prompt warrants re-running the safety suite.
FAQ
Are safety evals the same as alignment research?
No. Alignment research studies why models behave the way they do and how to change that; safety evals are the testing layer that checks the resulting behavior against a bar before and after deployment.
Who runs safety evals — the model developer or an outside party?
Both, increasingly. Internal evals happen throughout development; external evals from independent labs or auditors add a check the developer cannot easily grade in their own favor.
Can a small team run meaningful safety evals without huge resources?
Yes, for the categories relevant to their use case. A narrow deployment does not need every eval a frontier lab runs — it needs evals that match its actual tool access, user base, and risk surface.
Do safety evals slow down model releases?
They add time, but the alternative — shipping without them — shifts the cost of finding failures onto users instead of testers, which is a worse tradeoff for anything with real-world consequences.
Where to go next