{"post":{"slug":"tool-result-caching-2026","title":"Tool Result Caching: Not Fetching the Same Thing Twice","excerpt":"Agents re-fetch the same file, re-run the same query, and re-search the same term within a single run. Caching those results is the cheapest speedup available.","category":"ai","date":"August 17, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">L</span>ook at the trace of a long agent run and count the duplicate tool calls. The same configuration file read at step 3, step 11, and step 26. The same search query issued twice with slightly different reasoning around it. The same API endpoint hit four times because four different sub-tasks each needed the same lookup.\n\nNone of this is a bug. The agent has no memory of what it already fetched beyond what is in its context, and once earlier turns are compacted away it genuinely does not know. Caching tool results fixes the waste without changing the agent's behaviour at all.\n\n## What changed in 2026\n\n- **Longer runs made repetition more visible.** Agents doing dozens of steps repeat far more than single-turn assistants ever did.\n- **Compaction made it structural.** Once earlier context is summarised away, the agent loses the detail it would have needed to avoid re-fetching.\n- **Tool latency became the bottleneck.** As model calls got faster, slow tools — API lookups, database queries, page fetches — became the dominant contributor to run duration.\n- **Subagent fan-out multiplied it.** Several workers independently fetching the same shared resource is the common case in delegated architectures.\n\n## What is safe to cache\n\nThe classification is straightforward and worth making explicit rather than assuming.\n\n| Tool type | Cacheable | Scope |\n| --- | --- | --- |\n| Read a file | Yes | Run, unless the agent writes files |\n| Search a static corpus | Yes | Cross-run |\n| Query reference data | Yes | Time-bounded |\n| Fetch a web page | Yes | Short TTL |\n| Query live operational data | Careful | Very short or not at all |\n| Create, update, delete | Never | — |\n| Send a message or payment | Never | — |\n| Anything with side effects | Never | — |\n\nThe dividing line is whether the call is a question or an action. Questions can be cached; actions cannot, and caching one means the second invocation silently does nothing while reporting success.\n\nThe row that needs judgement is live operational data. Caching a stock level, a queue depth, or an account balance for the duration of a run means the agent may act on a value that has changed. Whether that is acceptable depends entirely on what it does next — reading it for a summary is fine, reading it to decide whether to place an order is not.\n\n## Scoping the cache\n\n**Run-scoped** is the sensible default. Cache within a single agent run, discard at the end. This captures the large majority of the benefit — most repetition happens inside one run — while avoiding every cross-session staleness question. It is also trivially safe to reason about.\n\n**Cross-run** caching is worth it for genuinely static content: documentation, reference data, embeddings, schema definitions. It needs a TTL and an invalidation story, which is real work.\n\n**Shared across subagents** is where the largest win usually sits. In a fan-out where ten workers each read the same shared context file, a cache visible to all of them turns ten fetches into one. Since subagents are typically spawned to work in parallel on related material, overlap is the norm — see [subagent architecture](/blog/subagent-architecture-explained-2026).\n\nThe key is the tool name plus its normalised arguments. Normalisation matters: `read(\"./config.json\")` and `read(\"config.json\")` are the same call and should hit the same entry, and argument ordering in a JSON object should not produce a miss.\n\n## What caching does not save\n\nAn important limitation people get wrong: **a cached result still goes into the context window.**\n\nIf the agent asks to read a file, the file's contents enter the prompt whether they came from the cache or the source. You have saved the latency and the external call. You have not saved the tokens, and in a long run the context cost of repeated large results may matter more than the fetch cost.\n\nAddressing that is a different technique — either the agent needs to not re-request things, or the earlier results need clearing from context. Context editing handles the second, and it composes well with caching: clear stale tool results from the window, and when the agent asks again, serve from cache cheaply. See [context compaction](/blog/context-compaction-explained-2026) for the adjacent mechanism.\n\n## Common mistakes\n\n- **Caching write operations.** The second call reports success and does nothing.\n- **No argument normalisation.** Equivalent calls miss, and the cache does little.\n- **Cross-run caching without a TTL.** Agents act on stale data with no indication.\n- **Caching live operational values.** A correct-looking number that is no longer true.\n- **Expecting it to reduce context.** It saves calls, not tokens.\n- **No cache metrics.** A hit rate near zero means your keys are wrong; nobody notices without measurement.\n- **Caching errors as results.** A transient failure becomes a permanent one for the rest of the run.\n\n## FAQ\n\n**How much does this actually save?**\nEntirely dependent on how repetitive your agent is, which varies enormously by task. Instrument first — count duplicate tool calls per run before building anything. If duplication is low, this is not your bottleneck.\n\n**Should the agent know a result was cached?**\nGenerally no; transparency is the point. Where freshness matters, exposing a timestamp on the result lets the model reason about staleness, which is better than hiding it.\n\n**Does this help with cost or latency?**\nBoth, in proportions that depend on the tool. An expensive third-party API call saves money; a slow internal query saves time. Model tokens are unaffected either way.\n\n**How does it relate to prompt caching?**\nDifferent layers entirely. Prompt caching reduces the cost of re-sending context to the model; this reduces the cost of re-fetching from tools. They compose — see [prompt caching](/blog/prompt-caching-explained-2026).\n\n## Where to go next\n\nFor the context-side saving this does not provide, read [context compaction](/blog/context-compaction-explained-2026). For where shared caching pays most, [subagent architecture](/blog/subagent-architecture-explained-2026), and for seeing the duplicate calls in the first place, [AI agent observability](/blog/ai-agent-observability-2026).","hue":201,"metaTitle":"Tool Result Caching for AI Agents Explained","metaDescription":"Why AI agents repeat tool calls within a run, how to scope a tool result cache safely, and which tools should never be cached.","keywords":["tool result caching","agent tool cache","llm tool call optimization","agent latency reduction","idempotent tool calls","cache invalidation agents","agent cost reduction","ai agent performance 2026"],"tldr":["**Agents repeat themselves constantly** — the same file read three times in a run is normal, not exceptional.","**Cache key on the arguments, scope on the tool's nature** — a read is cacheable, a write never is.","**Run-scoped caching is the safe default** — it avoids staleness across sessions while capturing most of the benefit.","**Cached results still consume context** — caching saves the call, not the tokens the result occupies.","**Skip:** caching anything whose value can change during the run. A stale result is worse than a slow one."],"readTime":"5 min read","wordCount":1008},"related":[{"slug":"semantic-caching-for-llms-2026","title":"Semantic Caching for LLMs: Reusing Similar Answers","excerpt":"Exact-match caching almost never hits on natural language. Semantic caching matches on meaning instead, which works well until it returns the wrong answer confidently.","category":"ai","date":"August 13, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":206},{"slug":"prefix-caching-explained-2026","title":"Prefix Caching: Not Recomputing the Same System Prompt","excerpt":"Every request starts with the same two thousand tokens of instructions, and by default the model processes them from scratch every single time. Caching that work is nearly free.","category":"ai","date":"January 1, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":197},{"slug":"llm-context-caching-2026","title":"LLM context caching in 2026: how it cuts cost and latency","excerpt":"Context caching lets an LLM reuse the compute it already spent on a prompt instead of reprocessing it every call. Here is how it actually works and when it saves real money.","category":"ai","date":"July 2, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":290}],"sameCategory":[{"slug":"agent-cost-attribution-2026","title":"Agent Cost Attribution: Finding Where the Tokens Went","excerpt":"A single monthly API bill tells you that agents are expensive. It does not tell you which feature, which customer, or which retry loop is responsible. That gap is fixable.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":220},{"slug":"human-in-the-loop-agents-2026","title":"Human-in-the-Loop Agents: Where to Put the Approval Gate","excerpt":"Approve everything and the agent is a slow autocomplete. Approve nothing and you have handed irreversible actions to a system that can be talked into things.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":197},{"slug":"reasoning-effort-controls-2026","title":"Reasoning Effort Controls: Tuning How Hard a Model Thinks","excerpt":"Fixed thinking budgets gave way to effort levels, and the default is rarely the right setting. Here is how the dial behaves and where each level actually belongs.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":210},{"slug":"retrieval-metrics-explained-2026","title":"Retrieval Metrics: Measuring the Half of RAG That Fails","excerpt":"When a RAG system answers badly, the retrieval usually failed before the model ever spoke. Measuring generation quality alone hides that entirely.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":215}],"newer":{"slug":"thread-1-4-explained-2026","title":"Thread 1.4 Explained: Fixing the Multi-Border Mess","excerpt":"Thread meshes worked until you had two border routers from different vendors. Here is what 1.4 fixes, what still needs vendor cooperation, and whether to upgrade.","category":"tech","date":"August 17, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":228},"older":{"slug":"ufs-4-1-storage-explained-2026","title":"UFS 4.1 Explained: Why Phone Storage Speed Matters","excerpt":"Two phones with identical processors can feel very different, and storage is often why. Here is what UFS 4.1 changes and when you would actually notice.","category":"tech","date":"August 17, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":244}}