AI token pricing looks simple on a pricing page — a dollar figure per million tokens — until you try to estimate what a real workload will cost and realize input and output are priced differently, tokens are not words, and several discount mechanisms only apply under specific conditions. Understanding the actual billing unit is the difference between a cost estimate that is roughly right and one that is off by five times once the app is in production.
What changed in 2026
- The input-output price gap widened on reasoning models. Models that generate internal reasoning tokens before a final answer bill for those hidden tokens too, which can make output cost dominate a bill in ways a simple prompt-response model would not.
- Batch and caching discounts became standard across providers, not a single vendor feature — most major APIs now offer some combination of asynchronous batch pricing and prefix caching.
- Tiered pricing by context length appeared on some models, where crossing a token threshold (for example, past 200k tokens of input) shifts the per-token rate for that call, not just the total tokens billed.
- Per-model price cuts became frequent enough that hardcoded cost estimates go stale fast — treat any specific dollar figure, including the ones in this post, as something to verify against current pricing pages before budgeting.
How tokenization actually works
A token is a chunk of text the model processes as one unit — often a common word, part of a longer word, or a piece of punctuation. English text runs roughly 1.3 tokens per word on average, but that ratio is not fixed: code, symbols, non-English languages, and rare words all tokenize less efficiently, sometimes using two or three tokens per word. This matters directly for cost, because billing counts tokens, not characters or words. A workload heavy in code or a non-English language can cost noticeably more per unit of content than the same amount of plain English prose.
What actually gets billed
| Cost component |
What it covers |
Typical price relationship |
| Input tokens |
Everything sent to the model: prompt, system message, retrieved context, conversation history |
Baseline rate |
| Output tokens |
Everything generated: the response, plus hidden reasoning tokens on reasoning models |
Usually 3-5x the input rate |
| Cached input tokens |
A repeated prefix served from cache instead of reprocessed |
Discounted below standard input rate |
| Batch requests |
Asynchronous calls processed without a real-time latency guarantee |
Discounted below standard synchronous rate |
| Tool/function definitions |
Schemas sent with every call that uses tools |
Billed as ordinary input tokens |
Why output cost dominates more often than people expect
Because output tokens cost several times more than input tokens, a workload that generates long responses — detailed reports, extensive code, multi-step reasoning — can spend most of its budget on generation even if the prompt itself is short. This is the opposite of what intuition suggests when people picture "a big document sent to the model" as the main cost driver. In practice, constraining output length and asking for concise, structured responses is often a bigger lever on total spend than trimming the prompt.
Estimating real cost before you build
Start by identifying the actual input and output token counts for a representative sample of real requests, not synthetic examples. Multiply by current per-token rates for input and output separately, then multiply by expected call volume. If a workload has variable difficulty, sending easy queries to a cheaper model with an LLM router often reduces the bill more than any single-model optimization. Do not extrapolate from a single test prompt; token counts vary enough across real user inputs that a small sample undercounts the tail.
FAQ
Is output really always more expensive than input?
On essentially every major provider, yes, and often by a wide margin. Some pricing pages list output at three to five times the input rate. Always check the current numbers for the specific model you are using rather than assuming a ratio.
Do longer context windows cost more per token?
Not directly on most providers — the per-token rate is usually the same regardless of how much of the context window you use, though some providers add a higher tier past a large token threshold. What increases is the total token count you are billed for, since a longer prompt is simply more tokens.
Does streaming change the price?
No. Streaming affects how the response is delivered, not how many tokens are billed. The cost is the same whether the response arrives all at once or token by token.
How much does reasoning add to a bill?
On reasoning-capable models, internal reasoning tokens are typically billed as output tokens even though the user never sees them directly, which can meaningfully increase cost for tasks that trigger long reasoning chains. Check the specific model documentation for how it reports and bills these tokens.
Where to go next