{"post":{"slug":"llm-unit-economics-2026","title":"LLM Unit Economics: What One User Actually Costs","excerpt":"Token pricing tells you nothing until you know how many calls one user action triggers. The gap between the two is where AI products discover they have negative gross margin.","category":"ai","date":"January 7, 2026","author":"ByteLedger Team","content":"<span class=\"dropcap\">T</span>oken prices are published and easy to compare. They are also nearly useless on their own, because the question that determines whether your product works is not what a token costs but what a *user* costs.\n\nThe gap between those is filled with fan-out, retries, evaluation traffic, and a usage distribution that is far more skewed than most people assume.\n\n## What changed in 2026\n\n- **Agentic features broke naive cost models.** One user action triggering dozens of model calls made per-token reasoning obviously inadequate.\n- **Usage-based pricing spread.** More AI products moved away from flat unlimited plans after encountering the tail.\n- **Caching became a first-order economic factor.** The difference between cached and fresh input tokens changed feature viability.\n- **Cost attribution matured.** Tagging spend by feature and customer became standard rather than aspirational — see [agent cost attribution](/blog/agent-cost-attribution-2026).\n\n## Build the unit from the action\n\nStart with a user-visible action — sending a message, generating a report, running an analysis — and count everything it triggers.\n\n| Component | Frequently counted | Often missed |\n| --- | --- | --- |\n| Primary model call | Yes | — |\n| Retrieval and embedding | Sometimes | — |\n| Reranking | Rarely | Yes |\n| Subagent or tool-loop calls | Rarely | Yes |\n| Guard model screening | Rarely | Yes |\n| Retries on failure | Rarely | Yes |\n| Evaluation sampling | Almost never | Yes |\n| Abandoned or cancelled work | Almost never | Yes |\n\nThe right column is where cost models go wrong. An agentic action with a handful of subagent calls, a reranker per retrieval, a guard model on input and output, and occasional retries can be an order of magnitude above the primary call everyone budgeted for.\n\nEvaluation traffic in particular scales with the traffic it measures, so it grows exactly when you least want a surprise — see [online evals](/blog/online-evals-for-llm-apps-2026).\n\n## The distribution is skewed\n\nAverage cost per user is a comforting and misleading number, because usage is not normally distributed.\n\nA small fraction of users generate a large share of consumption. In agentic products the skew is more pronounced, because a heavy user does not just make more requests — their requests tend to be larger, longer-running, and more likely to fan out.\n\nTwo implications. **Flat unlimited pricing is a bet on the tail**, and on an agentic product that bet is frequently lost. And **provisioning on average usage under-provisions for peak**, since the heavy users are also concentrated in time.\n\nModel the distribution rather than the mean. The numbers that matter are median cost per user, the cost at the high percentiles, and what fraction of total spend the top decile represents.\n\n## Caching changes the answer\n\nA cached prompt prefix costs a small fraction of fresh input. On a workload with a long stable system prompt and short user messages, that is the difference between viable and not.\n\nAgent loops benefit most, because each turn re-sends everything before it. Without caching, an agent loop's prompt cost grows roughly quadratically with turns; with it, most of each turn's prefix is cached — see [prefix caching](/blog/prefix-caching-explained-2026).\n\nWhich means a cost model built without caching can be wrong by a large multiple, in the favourable direction. It also means anything that breaks caching — a timestamp in the system prompt, non-deterministic tool ordering — has a direct and substantial cost consequence.\n\nBatch endpoints are the other large lever: roughly half price for anything with no user waiting, which covers evaluation, indexing, and enrichment — see [batch APIs](/blog/batch-api-for-llms-2026).\n\n## Common mistakes\n\n- **Reasoning in tokens rather than actions.** Wrong unit.\n- **Counting only the primary call.** Misses the majority in agentic products.\n- **Averaging over a skewed distribution.** Understates exposure.\n- **Ignoring evaluation traffic.** Scales with what it measures.\n- **Modelling without caching.** Wrong by a large factor.\n- **Flat pricing on agentic features.** Unbounded exposure to the tail.\n- **No per-feature attribution.** Cannot tell which feature is unprofitable.\n\n## FAQ\n\n**How do I find my real cost per action?**\nTag every model call with the user action that triggered it, then aggregate by trace. Without a trace ID linking fan-out back to one action, this is not reconstructable — see [agent cost attribution](/blog/agent-cost-attribution-2026).\n\n**What margin should I target?**\nThat is a business question rather than a technical one. The technical requirement is knowing the number, which most teams do not, and knowing how it moves with usage.\n\n**Should I cap usage?**\nSome limit is prudent on anything agentic, and it need not be visible — a per-user budget that almost nobody reaches protects against abuse and runaway loops without affecting normal users.\n\n**How do costs change as I scale?**\nPer-unit costs typically fall through caching hit rates improving with volume and through committed pricing. Total cost still grows with usage, so unit economics matter more at scale rather than less.\n\n## Where to go next\n\nFor measuring where spend goes, read [agent cost attribution](/blog/agent-cost-attribution-2026) and [cost per conversation](/blog/cost-per-conversation-2026). For the largest cost lever available, [prefix caching](/blog/prefix-caching-explained-2026).","hue":205,"metaTitle":"LLM Unit Economics: What One User Actually Costs","metaDescription":"How to calculate the real cost of an AI feature per user action, why usage distributions are skewed, and which costs are usually uncounted.","keywords":["llm unit economics","cost per user ai","ai gross margin","usage distribution","ai pricing model","cost per task","ai product economics","ai business model 2026"],"tldr":["**Cost per user action is the unit, not cost per token** — one action may trigger many calls.","**Heavy users are not average users** — the distribution is skewed, and flat pricing exposes you to the tail.","**Retries, evaluation, and abandoned work are real cost** — usually uncounted.","**Caching changes the arithmetic substantially** — a cached prefix costs a fraction of fresh input.","**Skip:** flat unlimited pricing on an agentic product until you have measured the tail."],"readTime":"4 min read","wordCount":853},"related":[{"slug":"ai-agent-cost-per-task-2026","title":"AI Agent Cost Per Task: The Number That Actually Matters","excerpt":"Cost per token tells you nothing about whether an agent is economical. Here is how to compute cost per successful task and what usually dominates the total.","category":"ai","date":"August 17, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":204},{"slug":"how-ai-companies-actually-make-money-2026","title":"How AI companies actually make money in 2026 — and which ones are burning cash","excerpt":"OpenAI, Anthropic, Google DeepMind, xAI, Mistral. We broke down where each one's revenue actually comes from, where the losses are hidden, and which businesses are real vs which are still set on fire by VC money.","category":"ai","date":"January 18, 2026","author":"ByteLedger Team","readTime":"11 min read","hue":195},{"slug":"how-to-reduce-ai-api-costs-2026","title":"How to reduce AI API costs in 2026: 9 levers that actually cut the bill","excerpt":"AI API bills balloon from waste, not usage. These nine levers — caching, model routing, shorter context — cut spend without hurting quality. Start with two.","category":"ai","date":"May 3, 2026","author":"ByteLedger Team","readTime":"3 min read","hue":158}],"sameCategory":[{"slug":"agent-cost-attribution-2026","title":"Agent Cost Attribution: Finding Where the Tokens Went","excerpt":"A single monthly API bill tells you that agents are expensive. It does not tell you which feature, which customer, or which retry loop is responsible. That gap is fixable.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":220},{"slug":"human-in-the-loop-agents-2026","title":"Human-in-the-Loop Agents: Where to Put the Approval Gate","excerpt":"Approve everything and the agent is a slow autocomplete. Approve nothing and you have handed irreversible actions to a system that can be talked into things.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":197},{"slug":"reasoning-effort-controls-2026","title":"Reasoning Effort Controls: Tuning How Hard a Model Thinks","excerpt":"Fixed thinking budgets gave way to effort levels, and the default is rarely the right setting. Here is how the dial behaves and where each level actually belongs.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":210},{"slug":"retrieval-metrics-explained-2026","title":"Retrieval Metrics: Measuring the Half of RAG That Fails","excerpt":"When a RAG system answers badly, the retrieval usually failed before the model ever spoke. Measuring generation quality alone hides that entirely.","category":"ai","date":"August 19, 2026","author":"ByteLedger Team","readTime":"5 min read","hue":215}],"newer":{"slug":"gin-indexes-explained-2026","title":"GIN Indexes: Indexing Values Inside a Column","excerpt":"A B-tree indexes a column's value. A GIN index indexes the things inside it — array elements, document keys, words in a text field — which is a different problem.","category":"code","date":"January 7, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":316},"older":{"slug":"model-souping-explained-2026","title":"Model Souping: Averaging Weights That Should Not Average","excerpt":"Averaging the weights of several fine-tuned models sounds like it should produce nonsense. Under specific conditions it produces something better than any of them.","category":"ai","date":"January 7, 2026","author":"ByteLedger Team","readTime":"4 min read","hue":207}}