Look at the trace of a long agent run and count the duplicate tool calls. The same configuration file read at step 3, step 11, and step 26. The same search query issued twice with slightly different reasoning around it. The same API endpoint hit four times because four different sub-tasks each needed the same lookup.
None of this is a bug. The agent has no memory of what it already fetched beyond what is in its context, and once earlier turns are compacted away it genuinely does not know. Caching tool results fixes the waste without changing the agent's behaviour at all.
What changed in 2026
- Longer runs made repetition more visible. Agents doing dozens of steps repeat far more than single-turn assistants ever did.
- Compaction made it structural. Once earlier context is summarised away, the agent loses the detail it would have needed to avoid re-fetching.
- Tool latency became the bottleneck. As model calls got faster, slow tools — API lookups, database queries, page fetches — became the dominant contributor to run duration.
- Subagent fan-out multiplied it. Several workers independently fetching the same shared resource is the common case in delegated architectures.
What is safe to cache
The classification is straightforward and worth making explicit rather than assuming.
| Tool type |
Cacheable |
Scope |
| Read a file |
Yes |
Run, unless the agent writes files |
| Search a static corpus |
Yes |
Cross-run |
| Query reference data |
Yes |
Time-bounded |
| Fetch a web page |
Yes |
Short TTL |
| Query live operational data |
Careful |
Very short or not at all |
| Create, update, delete |
Never |
— |
| Send a message or payment |
Never |
— |
| Anything with side effects |
Never |
— |
The dividing line is whether the call is a question or an action. Questions can be cached; actions cannot, and caching one means the second invocation silently does nothing while reporting success.
The row that needs judgement is live operational data. Caching a stock level, a queue depth, or an account balance for the duration of a run means the agent may act on a value that has changed. Whether that is acceptable depends entirely on what it does next — reading it for a summary is fine, reading it to decide whether to place an order is not.
Scoping the cache
Run-scoped is the sensible default. Cache within a single agent run, discard at the end. This captures the large majority of the benefit — most repetition happens inside one run — while avoiding every cross-session staleness question. It is also trivially safe to reason about.
Cross-run caching is worth it for genuinely static content: documentation, reference data, embeddings, schema definitions. It needs a TTL and an invalidation story, which is real work.
Shared across subagents is where the largest win usually sits. In a fan-out where ten workers each read the same shared context file, a cache visible to all of them turns ten fetches into one. Since subagents are typically spawned to work in parallel on related material, overlap is the norm — see subagent architecture.
The key is the tool name plus its normalised arguments. Normalisation matters: read("./config.json") and read("config.json") are the same call and should hit the same entry, and argument ordering in a JSON object should not produce a miss.
What caching does not save
An important limitation people get wrong: a cached result still goes into the context window.
If the agent asks to read a file, the file's contents enter the prompt whether they came from the cache or the source. You have saved the latency and the external call. You have not saved the tokens, and in a long run the context cost of repeated large results may matter more than the fetch cost.
Addressing that is a different technique — either the agent needs to not re-request things, or the earlier results need clearing from context. Context editing handles the second, and it composes well with caching: clear stale tool results from the window, and when the agent asks again, serve from cache cheaply. See context compaction for the adjacent mechanism.
Common mistakes
- Caching write operations. The second call reports success and does nothing.
- No argument normalisation. Equivalent calls miss, and the cache does little.
- Cross-run caching without a TTL. Agents act on stale data with no indication.
- Caching live operational values. A correct-looking number that is no longer true.
- Expecting it to reduce context. It saves calls, not tokens.
- No cache metrics. A hit rate near zero means your keys are wrong; nobody notices without measurement.
- Caching errors as results. A transient failure becomes a permanent one for the rest of the run.
FAQ
How much does this actually save?
Entirely dependent on how repetitive your agent is, which varies enormously by task. Instrument first — count duplicate tool calls per run before building anything. If duplication is low, this is not your bottleneck.
Should the agent know a result was cached?
Generally no; transparency is the point. Where freshness matters, exposing a timestamp on the result lets the model reason about staleness, which is better than hiding it.
Does this help with cost or latency?
Both, in proportions that depend on the tool. An expensive third-party API call saves money; a slow internal query saves time. Model tokens are unaffected either way.
How does it relate to prompt caching?
Different layers entirely. Prompt caching reduces the cost of re-sending context to the model; this reduces the cost of re-fetching from tools. They compose — see prompt caching.
Where to go next
For the context-side saving this does not provide, read context compaction. For where shared caching pays most, subagent architecture, and for seeing the duplicate calls in the first place, AI agent observability.