Train a model to refuse harmful requests and you will succeed. Train it too enthusiastically and you get a model that declines to explain how a lock works, discuss a medication's side effects, or help write a villain's dialogue — because each superficially resembles something it learned to refuse.
That is over-refusal, and it is the dominant practical failure in safety training. It is also harder to detect than under-refusal, because a refused legitimate request looks like the system working.
What changed in 2026
- Over-refusal became the measured concern. Benchmarks specifically targeting false refusals moved the field from optimising one direction to balancing two.
- Refusal quality improved. Explaining the boundary and offering an alternative displaced bare declines.
- The layered model settled. Base-model training for general safety plus a product-specific guard layer became the standard architecture.
- Refusal rate entered production monitoring. Tracking it as an operational metric caught silent regressions from prompt and model changes.
Two errors, asymmetric visibility
|
Under-refusal |
Over-refusal |
| Model does |
Helps with something harmful |
Declines something legitimate |
| Visibility |
High — gets reported |
Low — looks like working correctly |
| User impact |
Serious, rare |
Frustrating, frequent |
| Detection |
Red-teaming, reports |
Requires deliberate measurement |
The visibility asymmetry drives bad decisions. Under-refusals get escalated; over-refusals get a shrug and a user who stops using the product. Teams optimising against what they hear about systematically over-tighten.
Both need explicit measurement, and the over-refusal side needs a purpose-built evaluation set — legitimate requests that superficially resemble prohibited ones.
Near-miss examples calibrate the boundary
Training data with clearly harmful and clearly benign examples teaches a coarse distinction. The model learns to refuse on surface features — topic, vocabulary, framing — because those correlate in an easy dataset.
The examples that teach a usable boundary are pairs that look alike and differ in whether refusal is appropriate. Chemistry explained for a student versus synthesis instructions. Security concepts explained versus a working exploit. Medical information versus dosing advice for a specific person.
Those pairs force the model to attend to the actual distinction rather than to keywords. They are also considerably more expensive to produce, which is why datasets skew toward easy examples and models skew toward over-refusal.
What a good refusal looks like
A bare "I can't help with that" is the worst version. The user does not know what was objectionable, whether rephrasing would help, or whether the system is broken.
A useful refusal states what it will not do, briefly says why, and offers what it can do instead. That converts a dead end into a redirection, and it makes the boundary legible so the user can work within it.
It should also avoid moralising. A refusal that lectures reads as sanctimonious and generates more frustration than the refusal itself — a distinct failure from refusing at all, and one that shows up in user feedback about tone.
Monitoring in production
Refusal rate is an operational metric and belongs on a dashboard.
It moves for reasons unrelated to safety: a prompt change makes the model more cautious, a model upgrade shifts the boundary, a retrieval change surfaces context that triggers caution. None of these announce themselves, and a rising refusal rate is frequently the first signal that a change degraded the product.
Segment it. An overall rate hides a category where refusals jumped from near zero to substantial. And sample the refusals themselves periodically — reading what was actually declined is the only way to tell a correct boundary from an over-tightened one.
Adding refusal cases to your regression suite catches boundary shifts before deployment, per LLM regression testing.
Common mistakes
- Optimising only against under-refusal. Produces an unhelpful model.
- No over-refusal evaluation set. The failure is invisible without one.
- Training on easy examples only. Teaches keyword-based refusal.
- Bare refusals. Reads as broken.
- Moralising in the refusal. A separate and avoidable irritation.
- Not monitoring refusal rate. Silent regressions.
- Expecting training to encode product policy. Use a guard layer for rules specific to you.
FAQ
Can I reduce over-refusal with prompting?
Somewhat. A system prompt clarifying the legitimate scope helps, and it works against the model's trained behaviour rather than with it, so the effect is limited and inconsistent.
How do I handle product-specific rules?
A guard model, not training. Your rules about competitors, internal systems, or domain constraints are things the base model has never heard of — see guard models.
Does refusal behaviour change between model versions?
Frequently and without announcement. This is one of the more common ways a model upgrade silently changes product behaviour, which is why it belongs in pre-upgrade testing.
How do I measure over-refusal?
Build a set of legitimate requests that resemble prohibited ones, and track what fraction get declined. There is no substitute for constructing this deliberately.
Where to go next
For the product-specific layer, read guard models. For catching boundary shifts before release, LLM regression testing, and for the preference methods that shape this behaviour, DPO vs RLHF.