The first time an agent loop fails to converge, you find out from the billing dashboard. It ran for ninety minutes, made four hundred tool calls, rewrote the same file eleven times, and never decided it was finished. Nothing errored. The logs look busy and productive right up until you read them.
There are two ways to stop that, and they produce very different agents. One imposes a hard ceiling the model cannot see and does not anticipate. The other tells the model what it has to work with and lets it plan accordingly.
What changed in 2026
- Budgets became a request parameter rather than a wrapper you wrote. Tracking cumulative spend across loop iterations and aborting at a threshold used to be your job. It is now something the platform can enforce and, more usefully, communicate.
- The model became aware of its own budget. This is the substantive change. A countdown visible during generation lets the model decide to summarise rather than continue, to skip the optional verification pass, to write the conclusion now.
- Agentic pricing conversations shifted from per-token to per-task. "What does this cost per run" turned out to be the number teams actually needed, and it is not derivable from a token price without knowing loop behaviour.
- Runaway loops got recognised as a category. Early agent deployments treated non-convergence as a bug to fix. It is better understood as a permanent property to bound, the way you bound any retry.
Task budget vs max_tokens
|
Task budget |
max_tokens |
| Scope |
The whole agent loop |
One response |
| Model awareness |
Visible to the model during generation |
Invisible |
| Behaviour at the limit |
Agent wraps up and concludes |
Output truncates mid-token |
| Enforcement |
Advisory — the model paces itself |
Hard — the server stops emitting |
| Counts |
Generation plus tool results this turn |
Output tokens of this response |
| Failure mode |
Agent finishes shallow |
Agent finishes broken |
Both have a place, and they are not alternatives. max_tokens is a backstop that prevents a single pathological response from running forever. A task budget is a planning input that changes what the agent chooses to do. Ship an agent with only the former and every budget exhaustion produces a half-written sentence; ship with only the latter and a genuinely stuck model can still overrun.
Why visibility changes the output
It is tempting to treat a budget as pure accounting — a number that decrements until something stops. The interesting part is behavioural.
An agent that knows it has most of its budget left will explore: read more files, verify assumptions, try a second approach when the first looks shaky. An agent that knows it is nearly out will consolidate: stop gathering, write what it has, flag what it did not get to. Both are correct responses to the situation, and the model can only make the choice if it knows which situation it is in.
The practical consequence is that a budgeted agent produces different work, not just less of it. A run that ends with "I checked three of the five services; the remaining two are X and Y" is far more useful than one that ends mid-audit with no indication that anything was left. Getting that behaviour is mostly free — you set the number — but it does mean an undersized budget degrades quality in a specific direction rather than uniformly.
Sizing the number
There is no formula, but there is a method that converges quickly.
Instrument before you cap. Run the agent unbudgeted on twenty representative tasks and record actual consumption. You will find a distribution, usually with a long right tail — most tasks cluster, a few are genuinely large, and a couple are runaways that should never have been counted.
Set the budget near the top of the legitimate cluster, not at the mean and not at the max. Sizing to the mean fails half your real work; sizing to the observed maximum lets the runaway define your ceiling.
Watch the exhaustion rate. If more than a small fraction of runs hit the budget, it is too tight and you are systematically shipping shallow results. If nothing ever hits it, it is not doing anything and a pathological run will still hurt you.
Budget by task class, not globally. A code review and a full migration do not belong under the same number. Routing by task type before setting the budget is more effective than any amount of tuning a single value.
The per-task figures in AI agent cost per task are a reasonable starting reference if you have no baseline of your own yet.
Common mistakes
- Relying on max_tokens alone for loop control. It bounds one response, not the loop. An agent can make two hundred perfectly-sized calls.
- Setting the budget from a cost target rather than observed usage. "$0.50 per run" is a business constraint, not a task requirement. Measure first, then decide whether the work fits the budget or the budget needs to move.
- Passing a client-computed remaining figure while also resending full history. The server tracks the countdown itself; overriding it with your own arithmetic usually under-reports and truncates the agent early.
- Treating budget exhaustion as success. A run that completed within budget but stopped short is a partial result. It needs to be surfaced as one, not logged as a clean finish.
- No per-user or per-tenant aggregate. A per-run budget does not stop one user from triggering ten thousand runs.
- Forgetting the tool results count. Agents that read large files burn budget on input they did not generate, which surprises people who model spend purely on output.
FAQ
Does a budget make the agent worse?
A well-sized one makes it more predictable and slightly shallower at the margin. An undersized one makes it clearly worse. The tradeoff is real, which is why measuring the distribution first matters more than picking a clever number.
How does this interact with reasoning effort?
They compose. Effort controls how deeply the model thinks per step; the budget controls how many steps it can afford. Lowering effort is usually the better first lever for cost, because it reduces spend without reducing scope — see reasoning effort controls.
What is the difference between a task budget and a hard spending cap?
A task budget is a token-denominated hint to one loop. A spending cap is a currency-denominated limit enforced by the platform regardless of what the model wants. Production systems generally want both, at different layers.
Should the agent be told what to do when the budget runs low?
Yes, and it is underused. An instruction to summarise findings and state what remains unverified converts budget exhaustion from a truncation into a deliverable.
Where to go next
Budgets only help if you can see where the tokens went — agent cost attribution covers slicing spend by feature and user. For long sessions that hit context limits rather than cost limits, read context compaction, and for the broader picture of keeping inference spend sane, AI inference cost optimization.