The difference between an AI feature that prints money and one that bankrupts your startup is usually a 4-line config change. Most teams shipping AI features in 2026 are leaving 70–90% on the table by routing every request to the most expensive model, ignoring prompt caching, and never touching the batch API. This guide is the playbook for not doing that.
These eight strategies are how serious AI shops actually keep API costs sane in 2026. Each one is verified against current pricing as of April 2026 across OpenAI, Anthropic, and Google.
The pricing reality (as of April 2026)
Before strategies, the input/output cost-per-million-tokens you're optimising against:
| Model |
Input ($/M tokens) |
Output ($/M tokens) |
Best for |
| GPT-5 |
$5 |
$20 |
High-quality reasoning, frontier tasks |
| GPT-5 Mini |
$0.25 |
$1 |
Default for most workloads |
| Claude Opus 4.7 |
$15 |
$75 |
Best raw quality, longest reasoning |
| Claude Sonnet 4.6 |
$3 |
$15 |
Workhorse, near-Opus quality 80% of the time |
| Claude Haiku 4.5 |
$0.80 |
$4 |
Fast, cheap, surprisingly capable |
| Gemini 2.5 Pro |
$1.25 |
$10 |
Long context (1M+), multimodal |
| Gemini 2.5 Flash |
$0.075 |
$0.30 |
Cheapest credible option for high-volume |
| DeepSeek V3 |
$0.30 |
$0.90 |
Cheapest non-Google option |
| Llama 3.1 70B (Groq, Together) |
$0.60 |
$0.60 |
Fast inference, open weights |
Notice the spread: at the extremes, Opus output is 250x more expensive than Gemini Flash output. That's the lever. Pulling it correctly is what separates teams that scale on AI from teams that bleed cash.
Strategy 1 — Model routing (the biggest lever, by a mile)
Stop sending everything to Opus or GPT-5. Build a cheap router that classifies each request and dispatches to the cheapest model that can handle it.
A working pattern, in pseudocode:
async function routeRequest(userMessage: string): Promise<Response> {
// Step 1: ask a tiny model to classify
const classification = await classify(userMessage);
// Step 2: dispatch
switch (classification.complexity) {
case 'trivial': // 'what time is it?', 'thanks', single-word answers
return callGeminiFlash(userMessage);
case 'standard': // most normal queries
return callClaudeSonnet(userMessage);
case 'complex': // multi-step reasoning, math proofs, nuanced writing
return callClaudeOpus(userMessage);
}
}
The classifier itself uses a cheap model (Gemini Flash or GPT-5 Mini). One extra cheap call to save you 10–50 expensive calls. The math is overwhelming.
In our testing across a real customer-support workload of 10,000 messages:
| Routing strategy |
Cost per 1k messages |
Quality (human-rated) |
| All Opus |
$42 |
9.4/10 |
| All Sonnet |
$9 |
9.0/10 |
| All Haiku |
$2 |
7.8/10 |
| Smart routing (80/15/5 mix) |
$4 |
9.2/10 |
Smart routing gets you within 0.2 of pure-Opus quality at 10% of the cost. This is the single highest-ROI optimisation available.
Strategy 2 — Prompt caching (free money, almost always)
All three major providers now support prompt caching:
- OpenAI — automatic for repeated prompts ≥1024 tokens; cached input billed at ~50% of normal.
- Anthropic — explicit cache_control headers; cached input billed at 10% of normal (2.5x write cost on first call).
- Google — explicit context caching; cached input billed at ~25% of normal.
Where this matters: your system prompt (often 1k–10k tokens describing the AI's role, tone, constraints) gets sent on every single request. Without caching, you pay full price for it every time. With caching, you pay full price once per cache TTL window (usually 5 min to 1 hour) and ~10% thereafter.
For a chatbot doing 1,000 requests/hour with a 5,000-token system prompt:
- Without caching: 5,000 × 1,000 = 5M cached input tokens/hour × $3/M = $15/hour just on system prompt
- With Anthropic caching: ~$1.50/hour for the same workload
That's $13.50/hour, $324/day, $9,720/month saved on a single optimisation. Most teams ship without it.
Strategy 3 — Batch APIs (50% off, if you can wait)
OpenAI, Anthropic, and Google all offer batch APIs at ~50% of synchronous pricing. The catch: results return within 24 hours, not real-time.
Workloads that should use batch:
- Embedding generation for new documents
- Overnight summarisation of the day's customer tickets
- Bulk content moderation
- Periodic re-classification of historical data
- Background enrichment (e.g. tagging product catalogue)
- Dataset generation for fine-tuning
Workloads that can't:
- User-facing chat
- Anything in a request-response loop with a UI
- Real-time decisioning
A useful pattern: the same job runs synchronously when triggered by a user, batched when scheduled by a cron. Same code, different dispatch. 50% savings on the bulk path.
Strategy 4 — Structured outputs (saves tokens AND prevents bugs)
Free-form prose responses are token-expensive and parse-fragile. Structured outputs (OpenAI's response_format: json_schema, Anthropic's tool-use, Google's response_schema) make the model emit exactly the JSON shape you specified — no preamble, no markdown wrappers, no "Sure, here's the data:" cruft.
Token comparison for "extract email and phone from this text":
Free-form response (typical):
"Sure! Here's what I found:
- Email: alice@example.com
- Phone: 555-1234
Let me know if you need anything else!"
→ ~30 tokens
Structured output:
{"email":"alice@example.com","phone":"555-1234"}
→ ~12 tokens
That's a 60% output-token reduction on extraction-style tasks. Multiply by your call volume.
Beyond cost: structured outputs also eliminate an entire class of parsing bugs ("the model occasionally wraps the JSON in markdown" → no it doesn't, the API guarantees it now).
Strategy 5 — Choose the right context strategy
Long context is expensive — Gemini 2.5 Pro at $1.25/M input is cheap until you start sending 500K-token prompts every request, at which point your bill is one cent per call × 100K daily calls = $1,000/day.
Three strategies, ranked by efficiency:
- RAG (retrieval-augmented generation). Embed your docs once (cheap), retrieve top-k relevant chunks per query (cheap), only send those chunks to the LLM (cheap). Best for large knowledge bases.
- Prompt caching with selective context. If your context fits in a cache (<200K tokens for Claude, ~1M for Gemini) and you query it many times, cache it once and pay 10% thereafter.
- Naive long context. Send the whole document every time. Easiest. Most expensive. Use only when neither RAG nor caching fits.
Most teams default to (3) when (1) or (2) would cut costs 90% and improve quality (RAG over a 50-page manual is usually more accurate than stuffing the whole manual into context).
Strategy 6 — Set per-user daily limits (or someone WILL abuse you)
If you offer any free tier — even a paid tier with no per-user cap — you will eventually have a user who spends 18 hours a day chatting with your AI feature, costing you $20/day in API fees.
Implement two limits, day one:
- Per-user daily token cap (e.g. 100K input + 50K output tokens/day for free tier).
- Per-user request rate limit (e.g. max 60 requests/hour).
When a user hits the cap, gracefully degrade — show them their usage, explain the limit, offer to upgrade. Don't let a single user blow your monthly margin.
This is even more important for B2B free trials. A "free 14 days" with no usage cap = a single bad-actor signup costs you $500.
Strategy 7 — Embedding cost is real (and it's almost free)
Embeddings (vector representations of text used for RAG, semantic search, recommendations) are dramatically cheaper than chat completions. April 2026 pricing:
| Provider |
Embedding model |
$/M tokens |
| OpenAI |
text-embedding-3-small |
$0.02 |
| OpenAI |
text-embedding-3-large |
$0.13 |
| Anthropic |
(uses third-party — no native) |
— |
| Google |
text-embedding-005 |
$0.025 |
| Cohere |
embed-english-v4 |
$0.10 |
| Voyage |
voyage-3 |
$0.06 |
Embedding 1 million tokens of text costs about 2¢. Embed everything. Build proper search and RAG. Stop paying chat-completion prices for what should be a vector lookup.
A common antipattern: pasting an entire 100-page manual into ChatGPT to ask one question. Cost: ~$1 per query at Opus rates. Same workflow with embeddings + chat: ~$0.01 per query. 100x savings.
Strategy 8 — Fine-tuning is rarely worth it (in 2026)
If you're tempted to fine-tune a model to "save costs"... probably don't. Fine-tuning made sense in 2022. In 2026, between strong base models, prompt caching, and JSON-mode structured outputs, fine-tuning is rarely the cost-effective path.
When fine-tuning is still worth it:
- Very high-volume single-task workloads where shaving 30% off latency or token count compounds to real savings.
- Specific style/domain adaptation where prompt engineering keeps falling short (legal writing voice, medical terminology, code in a niche language).
- Privacy/compliance requirements that mandate keeping the fine-tuned model in your own infrastructure.
When fine-tuning is overkill:
- Most chat applications.
- Most extraction tasks (use structured outputs).
- Most summarisation tasks (use Sonnet or GPT-5 Mini with caching).
The newer reality: prompt engineering + caching + RAG + smart routing typically beats fine-tuning on cost-effectiveness for 90% of use cases.
Putting it all together — a real example
A customer-support AI handling 10,000 messages/day, with a 4,000-token system prompt and average 500-token user messages:
Naive setup (everything to Opus, no caching):
- Input: 10,000 × 4,500 tokens = 45M tokens × $15/M = $675/day
- Output: 10,000 × 800 tokens = 8M tokens × $75/M = $600/day
- Total: $1,275/day = $38,250/month
Optimised setup (smart routing 80/15/5, prompt caching, structured outputs):
- 80% Haiku: 8,000 × 4,500 × ($0.80 × 0.1 cached + $0.80 × 0.9) = ~$26/day input + ~$26/day output
- 15% Sonnet: 1,500 × 4,500 × ($3 × 0.1 cached + $3 × 0.9) = ~$18/day + ~$23/day output
- 5% Opus: 500 × 4,500 × cached pricing = ~$30/day + ~$30/day output
- Total: ~$153/day = $4,590/month
Savings: $33,660/month, ~88%. Same product, same user experience, same volume.
Common cost-optimization mistakes
- "We'll optimise once we have scale." No — by the time you have scale, the bill is already in the four-figures and you've burned cash unnecessarily. Build the routing layer day one.
- Routing to the most expensive model "to be safe." Quality differences between Sonnet and Opus on most workloads are smaller than the marketing implies. Test rigorously, not pessimistically.
- Ignoring prompt caching because "it's a small optimisation." It's a 10x cost cut on cached input. Stop ignoring it.
- Sending the entire conversation history on every turn when only the last 3 messages matter for the current question. Smart context truncation > naive append.
- Re-embedding documents on every change instead of versioning chunks and only re-embedding what changed.
- Using a vector DB you don't need. For <100k documents, Postgres + pgvector or even a flat file with cosine similarity is fine. Pinecone/Weaviate/Qdrant are overkill until they're not.
- Underestimating output tokens. Output is 4–5x more expensive than input on most providers. Telling the model "be concise" + setting
max_tokens to a sensible cap is free money.
FAQ
What's the cheapest credible model in 2026?
Gemini 2.5 Flash at $0.075/M input is the cheapest from a major provider. DeepSeek V3 ($0.30/M) is close on quality at a slightly higher price with different jurisdiction trade-offs. Llama 3.1 70B on Groq/Together is competitive for self-hosted-feel workloads.
Is OpenAI's batch API really 50% off?
Yes — explicitly half-price for input and output. The catch is the 24-hour SLA. Use it whenever your workload allows.
How much does prompt caching actually save?
Anthropic: ~10% of normal input cost on cached tokens (90% saving). OpenAI: ~50% saving. Google: ~75% saving. The catch is each provider's TTL: Anthropic 5 min default (extendable to 1hr), OpenAI ~5–10 min, Google up to 1 hour.
Can I run my own model and skip API costs entirely?
You can — Llama 3.1, Qwen, DeepSeek all release competitive open weights. The breakeven vs API is somewhere around $5–10k/month of API spend, after which a self-hosted GPU cluster (Hetzner, Latitude, or AWS spot) starts to pencil out. Below that: use APIs, don't build infra.
Is "switching providers" a real cost lever?
Sometimes. Gemini Flash is meaningfully cheaper than OpenAI/Anthropic equivalents for high-volume cheap-model work. DeepSeek is cheaper still. But provider-lock for advanced features (function calling, structured output, multimodal) is real — switching isn't a one-line change.
What about Anthropic's "extended thinking" mode — is it worth the cost?
Charged at output rate per thinking token, so a 2,000-token thinking process is ~$0.15 on Opus. Worth it for genuinely hard reasoning tasks (math, complex code review). Not worth it for "what's the capital of France" — turn it off for trivial routing.
Should I cache user messages too?
Generally no — user messages vary too much to benefit from cache hits. Cache the system prompt and any large context (knowledge base passages). The user-message turn is small and unique.
How often should I re-evaluate my model routing?
Once per major model release (every 1–3 months in 2026's pace). What was the cheapest credible model 3 months ago may not be today.
The bottom line
The eight strategies above are how teams shipping serious AI features in 2026 stay solvent. None of them require deep ML expertise — they're configuration and architecture decisions, not model training. Smart routing alone typically saves 70%+; layering caching + structured outputs + batch on top adds another 20–30 points. Most teams operate at 10x the cost they should be at because they shipped the prototype's model: "claude-opus-4-7" straight to production. Don't be that team.
Related reading