Connect a handful of MCP servers to an agent and something quietly goes wrong. Each server contributes its tools, each tool contributes a name, a description, and a JSON schema, and all of it is sent on every single request before the model reads a word of the actual conversation.
Two hundred tools is not unusual once several integrations are attached. At that point the tool definitions can occupy more of the context window than anything the user said, you are paying for them on every turn, and — more damaging — the model is worse at picking the right one.
What changed in 2026
- MCP made tool sprawl easy. Connecting a server is a config line, and each one brings its full catalogue whether you need it or not.
- Deferred loading arrived as a first-class feature. Marking tools as deferred and providing a search tool became a supported pattern rather than something you hand-rolled.
- Selection accuracy got measured properly. The finding that mattered: quality degrades from tool count well before context limits bind, which reframed this as a reasoning problem rather than a capacity one.
- Tool curation became a design discipline. Teams started treating the tool set as something to design rather than accumulate.
Where the cost actually lands
| Cost |
When it bites |
Fixed by |
| Context consumed by definitions |
Every request |
Deferred loading |
| Tokens billed for those definitions |
Every request |
Deferred loading, caching |
| Model picks the wrong tool |
Grows with count |
Curation, better descriptions |
| Latency from a search round-trip |
Only with deferred loading |
Not deferring small sets |
| Cache invalidation on tool change |
Whenever the set changes |
Stable tool ordering |
The third row is the one people underestimate. The intuitive worry is running out of context; the practical problem arrives earlier. A model choosing among eighty tools with overlapping descriptions — search_docs, find_document, query_knowledge_base, lookup_article — makes worse choices than one choosing among eight distinct ones. Adding tools past a point makes the agent less capable, not more, and no amount of context headroom fixes that.
How deferred loading works
Instead of sending every schema, you mark most tools as deferred. The model receives a lightweight index — names and brief descriptions — plus a search tool that is not deferred. When it needs something, it searches, gets back the full definitions for matching tools, and calls one.
Two search flavours are typical: regex matching for when you know the naming conventions, and BM25-style keyword ranking for fuzzier lookup. Either works; the choice matters less than tool naming, since both search over text you wrote.
The constraint that trips people up: at least one tool must remain non-deferred, and the search tool itself must never be deferred. Defer everything and the agent has no entry point — an error, not a subtle degradation, but an easy configuration mistake to make when bulk-marking a catalogue.
The tradeoff is a round-trip. Finding a tool now costs a model call before the real work starts. Under roughly twenty tools that overhead exceeds what you saved, which is why this is a large-tool-set technique rather than a general optimisation.
Curation beats mechanism
Deferred loading solves the context cost. It does not solve the selection problem, and the selection problem is usually what is actually hurting you.
The highest-value work is reducing and clarifying the tool set:
Remove what nothing uses. Attached MCP servers frequently contribute tools no workflow calls. Every one is a distractor competing for selection.
Merge near-duplicates. Three tools that search slightly different things should usually be one tool with a parameter. The model is much better at filling a parameter than at distinguishing three similar descriptions.
Write descriptions that say when not to use the tool. A description covering the negative case — "use this for internal docs, not public web content" — disambiguates far better than one describing only capability.
Name for distinctness. search_internal_wiki beats search when four other tools also search things.
A curated set of fifteen good tools outperforms a deferred-loaded set of two hundred mediocre ones, and it costs less. Reach for the mechanism when the catalogue is genuinely large and genuinely necessary — tool calling reliability covers the description-writing side in more depth.
Common mistakes
- Deferring every tool. No entry point; the API rejects it.
- Using it on a small tool set. The search round-trip costs more than the definitions.
- Deferring instead of curating. Hides the bloat rather than fixing the selection problem underneath.
- Reordering tools between requests. Tool definitions sit early in the prompt, so a changed order invalidates the cache prefix and costs more than the tools ever did.
- Vague names and descriptions. Search only works as well as the text it searches.
- Adding every MCP server available. Each one is a permanent contribution to the selection problem.
FAQ
How many tools is too many?
Selection quality starts degrading noticeably in the low tens for similar tools, and much later for genuinely distinct ones. The real signal is your own error rate — if the agent picks wrong, you are past the limit regardless of the count.
Does prompt caching help instead?
It removes most of the token cost of a stable tool set, which is real relief, but the definitions still occupy context and the selection problem is untouched. Caching and deferred loading address different halves; caching is the easier win if your tool set is stable.
Can I load tools conditionally in my own code?
Yes, and it is often better. If you know from the request type that only a subset is relevant, filter before the call. That beats both approaches — no search round-trip, no wasted context, no selection ambiguity. Deferred loading is for when you genuinely cannot predict.
Does this work with subagents?
It composes well. Giving each subagent a small task-specific tool set is arguably the cleanest solution to tool sprawl, since each context stays focused — see agentic workflows.
Where to go next
For writing tool descriptions the model interprets correctly, read tool calling reliability. For the context budget this competes for, context engineering, and for the security implications of a large tool surface, AI agent permissions.