When an agent calls the wrong tool or passes a made-up argument, the instinct is to edit the system prompt. That is usually the wrong lever. The model selects from the tool definitions it was given, and those definitions are the highest-leverage prompt in the entire system — far more so than the instructions above them.
Most tool-calling problems are tool design problems.
What changed in 2026
- Tool count effects got quantified. Measurement confirmed what practitioners suspected: selection accuracy declines noticeably as the available tool list grows.
- Dynamic tool filtering spread. Rather than exposing everything, systems began selecting a relevant subset per request based on the task.
- Error message design got attention. Returning actionable, model-readable errors rather than raw exceptions became recognized as a meaningful accuracy improvement.
- Task-shaped tools beat endpoint mirrors. Consensus formed that tools should represent things a user wants done, not a one-to-one mapping of an existing API.
What drives selection accuracy
| Factor |
Effect |
| Number of tools available |
Accuracy declines as the list grows |
| Overlap between tools |
Largest single cause of wrong selection |
| Description specificity |
Vague descriptions produce guesses |
| Parameter naming |
Ambiguous names produce invented values |
| Required versus optional parameters |
Too many required fields cause fabrication |
| Error message quality |
Determines whether the model recovers or loops |
| Examples in the description |
Meaningfully improves argument formatting |
The overlap row is worth dwelling on. If you expose both a general search tool and a specialized document lookup, and both descriptions could plausibly cover the same request, the model will pick between them inconsistently. Fixing this means either merging them or rewriting descriptions so the boundary is unmistakable — stating explicitly when not to use each one.
Designing tools the model uses well
Write descriptions for a competent new colleague who has no context. Say what the tool does, when to use it, when not to use it, and what it returns. The negative case is the part everyone omits and the part that most improves selection when tools are similar.
Keep required parameters minimal. Every required field is something the model must produce, and if the information is not available it will invent something rather than fail. Make fields optional where the tool can sensibly default them.
Return errors the model can act on. A tool that fails with an internal exception message gives the model nothing to work with, so it retries identically and loops — the step-count explosion described in AI agent cost per task. A tool that returns "the date parameter must be in YYYY-MM-DD format, received 'next Tuesday'" gets a corrected call immediately.
Filter the tool list per request where you can. If the task is clearly about calendars, do not also offer eight database tools. A cheap classification step narrowing the tool set before the main call improves accuracy more than most prompt engineering.
And test it. Tool selection is measurable — build a set of requests with known-correct tool choices and score it, exactly as described in prompt A/B testing.
Common mistakes
- Mirroring your REST API as tools. Endpoints are shaped for programs; tools should be shaped for tasks.
- Twenty-plus tools in one call. Selection accuracy suffers well before that.
- Descriptions that only say what the tool does. Say when not to use it too.
- Raw exceptions as tool results. The model cannot act on a stack trace.
- All parameters required. Forces fabrication when information is missing.
- Fixing selection problems in the system prompt. The tool definitions are the stronger lever.
FAQ
How many tools is too many?
Accuracy starts degrading noticeably in the low double digits for most models. If you need more, filter dynamically per request rather than exposing everything.
Should tool results be structured or prose?
Structured for data the model will use programmatically, with a brief prose summary when the result needs interpretation. Both work; consistency matters more.
Does the model see my parameter descriptions?
Yes, and they materially affect argument quality. Describe formats and give an example value for anything with a specific shape.
What about tools that modify data?
Those need a permission layer independent of the model's judgment — see AI agent permissions.
Where to go next
For the underlying mechanism, read function calling explained. For output shape guarantees, JSON mode vs tool calling, and for containing what tools can do, AI agent permissions.