Every AI API bill comes down to counting tokens, not words, and charging a different rate for tokens you send versus tokens the model generates back. A token is a chunk of text — roughly four characters of English on average, though code, non-English languages, and unusual formatting all tokenize less efficiently. Understanding this explains almost every surprise on an AI bill: why a verbose model response costs more than the prompt that triggered it, why a document-heavy prompt is cheaper than it looks once caching kicks in, and why two requests with the same word count can cost noticeably different amounts.
How it works
A tokenizer breaks text into pieces before a model ever sees it — sometimes a whole word, sometimes a fragment of one, sometimes a single punctuation mark. English prose tokenizes efficiently, at roughly four characters per token or about three-quarters of a word per token. Code, tables, and non-English text typically tokenize less efficiently, meaning the same amount of visible content produces more tokens and therefore more cost.
Every API call has two token counts that matter for billing:
- Input tokens — your prompt, any system instructions, conversation history, and retrieved context you send with the request.
- Output tokens — everything the model generates in response.
These are priced separately, and output is reliably the more expensive side — generation happens sequentially, one token informing the next, which is more compute-intensive per token than processing an input prompt that can largely be handled in parallel. A response that rambles costs measurably more than one that answers directly, which is a real reason to instruct a model toward concise output beyond just readability.
What actually drives your bill
| Factor |
Effect on cost |
Practical lever |
| Input length |
Linear — more tokens in, more cost |
Trim unnecessary context, summarize history |
| Output length |
Linear, at a higher per-token rate |
Instruct for concise answers, cap max tokens |
| Repeated context |
Can be heavily discounted |
Use prompt caching for stable system prompts |
| Real-time vs delayed |
Batch is typically cheaper |
Move non-urgent jobs to a batch endpoint |
| Model tier |
Large multiplier |
Route easy tasks to a smaller, cheaper model |
Context window size itself is a red herring in cost conversations — a model with a huge available window costs you nothing extra unless you actually fill it. What you pay for is tokens used in a specific request, not capacity you did not touch.
How to estimate your own bill
- Count a representative sample. Run your actual prompts and typical responses through a tokenizer (most providers publish one) rather than guessing from word count.
- Separate input and output token counts. Multiply each by its respective rate — do not use a single blended per-token price, since the two differ substantially.
- Account for caching if your prompt has a stable portion. A system prompt or reference document repeated across calls should be counted at the discounted cached rate after the first call, not the full input rate every time.
- Multiply by realistic call volume, not peak-day volume, then sanity-check against a worst-case multiplier for traffic spikes.
- Re-check monthly. Usage patterns and vendor pricing both drift, and a bill estimate from launch can be stale within a quarter.
Common mistakes
Estimating cost from word count instead of tokens. The conversion ratio is not exact and varies by content type, so word-count-based estimates are reliably off, sometimes by a large margin for code-heavy workloads.
Forgetting output costs more than input. Teams obsess over trimming their prompt while ignoring that the model's response — priced at a multiple of the input rate — is often the larger share of the bill.
Not using caching for a repeated system prompt. Sending the same lengthy instructions at full price on every single call is a common, easily fixed waste once a provider offers cached input pricing.
Treating context window size as a cost factor. A large available window is a capability, not a cost — you are billed for tokens you actually use in a request, not the maximum the model could theoretically hold.
FAQ
How many tokens is a word, roughly?
Roughly three-quarters of a token per word for typical English prose, though this varies with vocabulary and formatting.
Why does output cost more than input?
Generating each output token depends sequentially on everything before it, while an input prompt can largely be processed in parallel — making generation more compute-intensive per token, which providers reflect in pricing.
Does a bigger context window cost more automatically?
No. You pay only for tokens actually included in a given request. A large context window is a ceiling on what you can send, not a charge for having the option available.
Is prompt caching worth setting up for a small app?
If your system prompt or context is short and calls are infrequent, the savings may be marginal. Once a stable prompt is being sent on a meaningful number of calls per day, caching typically pays for the setup effort quickly.
Where to go next
For real numbers across vendors, see AI API cost comparison 2026. If you are calling more than one provider, best AI API gateways in 2026 covers tools that track this spend for you, and vLLM vs Ollama in 2026 covers the self-hosted alternative to per-token billing entirely.