Generation ends one of three ways: the model emits an end-of-turn token, the output reaches the token limit, or it produces a string you nominated as a stopping point. The third is the one you control directly, and it is genuinely useful for structured extraction and for older-style formats.
It is also a common source of silent bugs, because the different ways of stopping are indistinguishable if you only look at the text.
What changed in 2026
- Structured outputs reduced the need substantially. Schema-constrained generation terminates correctly by construction, removing the main reason people reached for stop sequences.
- Native tool calling removed another. Parsing tool calls out of text with stop markers became unnecessary once structured tool-call formats arrived.
- Stop reason checking got emphasised. The failure mode of treating a truncated response as complete became better documented.
- They remained useful for specific formats. Extraction tasks and delimiter-based outputs still benefit.
The three stop reasons
| Reason |
Means |
Your response |
| End of turn |
Model finished naturally |
Use the output |
| Max tokens |
Hit the limit, output is truncated |
Retry with a higher limit, or handle partial |
| Stop sequence |
Your marker was produced |
Use the output |
| Refusal |
The model declined |
Different handling entirely |
The critical one is max tokens. A response cut off mid-sentence is still a string, and code that reads only the text treats it as a complete answer. In an extraction pipeline that means a partial record written to your database with no indication anything was wrong.
Always check the stop reason before using the output. This is a two-line change that prevents a whole class of silent data corruption, and it is skipped constantly.
Is the sequence included?
Behaviour varies by provider, and getting it wrong breaks parsing in ways that look mysterious.
Most implementations exclude the stop sequence from the returned text — generation stops and what you get is everything before it. Some include it. Some let you configure the behaviour.
Check for your specific provider rather than assuming, and write parsing that tolerates both where you can. If you are splitting on a delimiter that may or may not be present in the returned string, handle both cases.
Choosing a sequence
The requirement is that it cannot appear in legitimate output. A stop sequence that occurs naturally truncates valid responses, and the resulting bug is intermittent and confusing — most responses are fine, some are mysteriously short.
Poor choices: common words, single newlines, punctuation, anything that appears in ordinary prose.
Better choices: distinctive multi-character markers that would never occur naturally, ideally ones you also instruct the model to emit explicitly.
The pairing matters — a stop sequence works best when the prompt tells the model to produce that marker when finished. Then the sequence is a mechanism enforcing an instruction the model already understands, rather than an external cut-off.
Note also that most providers accept several stop sequences, and matching is generally on exact strings including whitespace. A sequence with a trailing space will not match output without one.
When not to use them
For length control. Use the maximum token parameter. Stop sequences terminate on structure, not on length, and using them to approximate a length limit works badly.
For reliable structured output. Schema-constrained generation is strictly better — it guarantees valid output and terminates correctly without delimiter parsing. If you are using stop sequences to extract JSON from a response, structured outputs solve the problem properly — see structured outputs.
For tool calls. Native tool-calling formats have made stop-marker parsing obsolete for this purpose.
Where they remain genuinely useful: extraction tasks with a repeated delimiter format, generation that should stop before an example the model tends to continue into, and interoperating with formats that predate structured output support.
Common mistakes
- Not checking the stop reason. Truncated responses treated as complete.
- Assuming the sequence is included or excluded. Provider-dependent.
- A sequence that appears in valid output. Intermittent truncation.
- Using them for length control. Max tokens is the right parameter.
- Using them where structured outputs apply. Weaker guarantee, more parsing.
- Whitespace mismatches. Exact matching means trailing spaces matter.
- Not instructing the model to emit the marker. Relying on it appearing by chance.
FAQ
How many stop sequences can I use?
Providers typically allow several, with a limit. More sequences means more chances one appears in valid output, so use the fewest that work.
What if the model never emits my sequence?
Generation continues to the token limit. That is why the stop reason check matters, and why instructing the model to produce the marker is part of the design rather than optional.
Do they save cost?
Marginally, by stopping generation earlier than it would otherwise. Not a meaningful cost lever compared with prompt length or caching.
Are they available with structured outputs?
Usually, and they are typically unnecessary there — schema enforcement handles termination.
Where to go next
For the better approach to structured generation, read structured outputs. For the parameters governing token choice, sampling parameters, and for validating what comes back, LLM output validation.