Before a token is sampled, the model produces a score for every option in its vocabulary. Logit bias lets you add a value to specific ones — positive to make them more likely, negative to make them less, large negative to make them effectively impossible.
It is the most precise output control available and the most fragile, because it operates on the model's tokenisation rather than on anything a person would recognise as a word.
What changed in 2026
- Structured outputs displaced most uses. Schema-constrained generation handles the common cases far more robustly.
- Support became inconsistent. Some current models restricted or removed the parameter, so portability is no longer assumable.
- Tokenizer differences got wider. As models diverged in tokenisation, biases stopped porting between them at all.
- The classification use case persisted. Constraining output to a small set of valid labels remained genuinely useful.
Tokens are not words
This is the source of nearly every logit bias bug.
A word may be one token or several, and the tokenisation depends on context — leading whitespace, capitalisation, and surrounding characters all change how a string is split. The word at the start of a sentence and the same word mid-sentence are frequently different tokens.
So biasing "the token for a word" is rarely a single operation. To suppress a word you would need to bias every token that could begin it, in every casing and whitespace variant, and even then the model can produce it by an unusual split.
The consequence: logit bias cannot reliably ban a word, let alone a concept. It can ban a token. Using it for content control produces a system that blocks one spelling while the model happily generates a paraphrase.
Where it works well
The clean case is constraining output to a small closed set.
For a classification returning one of five labels, you can bias the first token of each valid label upward and everything else down, guaranteeing the output starts with a valid option. That is a genuine reliability improvement for a task where the model occasionally editorialises instead of answering.
Similar narrow uses: forcing a yes/no answer, suppressing a specific token that the model repeatedly produces incorrectly in a structured format, and steering away from a particular formatting artefact.
What these have in common is a small, closed, known set operating at a known position. Where the target set is open-ended or the position varies, logit bias is the wrong tool.
| Use case |
Suitable? |
| Constrain to five known labels |
Yes |
| Force yes/no |
Yes |
| Suppress one specific formatting token |
Yes |
| Ban a word from output |
No — token boundaries |
| Enforce a content policy |
No — paraphrase defeats it |
| Guarantee valid JSON |
No — use structured outputs |
Portability and brittleness
Token IDs are model-specific. A bias configuration built for one model applies different biases, to different tokens, on another — silently, since the IDs are still valid numbers.
That makes logit bias configurations a hidden coupling to a specific model version. Changing models means recomputing every ID, and forgetting to do so produces subtly wrong behaviour rather than an error. Anything using it should record which model and tokenizer the IDs were derived from, and validating them should be part of a model migration checklist — see AI model deprecation planning.
Where a structured output or a constrained decoding grammar can express the same constraint, prefer it. Those operate at the level of valid output rather than valid tokens, which is both more robust and more portable.
Common mistakes
- Treating tokens as words. The fundamental misunderstanding.
- Using it for content policy. Paraphrase defeats it immediately.
- Not recording the source model. IDs silently mean something else elsewhere.
- Biasing without checking tokenisation. Verify what your string actually tokenises to.
- Using it where structured outputs apply. Weaker and more fragile.
- Extreme positive bias. Can force a token so strongly that output becomes nonsense.
FAQ
How do I find token IDs?
With the tokenizer for the specific model, encoding the exact string including leading whitespace. Do this programmatically rather than by hand, and re-derive it whenever the model changes.
Can I ban a phrase?
Not reliably. Multi-token phrases require biasing each token, which also suppresses those tokens in unrelated legitimate contexts. Post-generation filtering is more appropriate — see guard models.
Is it available everywhere?
No, and support has become inconsistent. Some current models reject it. Check before designing around it.
What is the alternative for valid output?
Structured outputs with schema enforcement, which guarantees the shape and is portable across models — see structured outputs.
Where to go next
For the robust alternative, read structured outputs. For controlling termination, stop sequences, and for content filtering done properly, guard models.