Prompt engineering in 2026 is a more targeted skill than it was in 2023. Frontier models handle more intent implicitly, structured output is now a first-class API feature, and reasoning models actively resist over-specified prompts. What remains is a small set of high-ROI techniques worth knowing precisely.
What changed in 2026
- Reasoning models changed the CoT calculus. With o3, Claude 3.7 extended thinking, and Gemini 2.5 Pro, you provide the goal and constraints — not the reasoning steps. Spelling out the reasoning process often hurts performance.
- JSON mode and tool schemas replaced output formatting prompts. "Return your answer as JSON with keys X, Y, Z" is now
response_format={"type": "json_schema", ...}.
- System prompts got shorter and more effective. Verbose 2,000-word system prompts routinely perform worse than 200-word focused ones.
- Prompt caching changed economics. Static system prompts are now cached on most providers; you pay full price only on the first call per session.
Techniques that still deliver
Chain-of-thought (CoT)
Adding "think through this step by step before answering" still improves accuracy significantly on multi-step problems, math, and code. For standard (non-reasoning) models, explicit CoT elicitation is one of the highest-ROI prompting techniques available.
Analyse whether this customer qualifies for a refund.
Think through the relevant policy clauses step by step,
then state your conclusion and reasoning.
Few-shot examples
For format and classification tasks, 3–5 examples of the exact input-output pattern you want outperform long prose instructions.
Examples:
Input: "My invoice total is wrong" → Category: billing
Input: "I cannot log in" → Category: auth
Input: "The app is slow" → Category: performance
Now classify: "The payment button does not work"
Role and context anchoring
Specifying who the model is and what it knows helps more than repeating constraints.
You are a senior Python engineer reviewing pull requests for a financial services company.
You care about correctness, security, and minimal diff size.
Constraint-first, not constraint-last
Placing constraints before the content to process is more reliable than appending them:
# Better
Rules: respond in 3 bullet points, no jargon, under 80 words.
Summarise: [content]
# Worse
Summarise: [content]
Rules: respond in 3 bullet points, no jargon, under 80 words.
Structured output in 2026
Do not write output format instructions when the API supports native structured output:
import anthropic, json
from pydantic import BaseModel
class Ticket(BaseModel):
category: str
priority: str
summary: str
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=256,
tools=[{
"name": "classify_ticket",
"description": "Classify a support ticket",
"input_schema": Ticket.model_json_schema(),
}],
tool_choice={"type": "tool", "name": "classify_ticket"},
messages=[{"role": "user", "content": "My payment failed on checkout."}],
)
ticket = Ticket(**response.content[0].input)
The schema enforces structure; the model never needs to be told to "respond as JSON."
Prompting reasoning models differently
For o3, Claude 3.7 extended thinking, and Gemini 2.5 Pro:
| Standard model |
Reasoning model |
| "Step 1: identify... Step 2: compare..." |
"Return the better approach and why" |
| Explicit reasoning chain |
Goal + constraints only |
| "Think step by step" |
Usually unnecessary — hurts CoT budget |
| Long system prompt with all rules |
Short system prompt, key constraints only |
Reasoning models have an internal thinking process; over-specifying the external reasoning steps constrains it.
System prompt best practices
- Under 300 words unless your task genuinely requires extensive domain context.
- One clear job description — what the model is, who it serves, what it never does.
- Constraints as bullet list, not prose paragraphs.
- No padding — "You are a helpful assistant who always tries to..." is not a constraint.
How to pick
- Is the task multi-step reasoning? Use CoT on standard models; give goal + constraints on reasoning models.
- Is the output structured? Use tool-calling schemas, not prose formatting instructions.
- Is the format complex but repeatable? Use 3–5 few-shot examples.
- Is quality inconsistent? Log 50 failures, find the pattern, add one targeted constraint.
- Is the system prompt over 400 words? Cut it — length is not a proxy for quality.
Common mistakes
Stacking conflicting instructions. "Be concise but comprehensive" gives the model no signal. Pick one.
Updating prompts without logging the change. You cannot know if the new prompt is better without comparing on the same set of examples.
Using CoT on reasoning models. "Think step by step" wastes the extended thinking budget on explicit narration instead of actual reasoning.
Never testing the prompt. Write the prompt, run it against 20 real examples, measure. Intuition is not enough.
What to skip
- "DAN" and jailbreak-style prompts in production — they are fragile and model-version-dependent.
- Multi-layer prompt chaining where one model call with a well-structured prompt does the job.
- Temperature tuning without evals — adjusting temperature without a score function is a random walk.
FAQ
Does prompt engineering still matter with better models?
Yes — but the surface area shrank. CoT, few-shot, and structured output matter. Arcane tricks mostly do not.
How do I measure if my prompt improved?
Define a test set with known correct outputs. Score the baseline prompt and the new prompt on the same set. Compare quantitatively.
What is prompt caching and should I use it?
Prompt caching stores the computed KV state for your system prompt prefix. Supported on Anthropic (cache_control), OpenAI, and Google. Use it whenever your system prompt is static and requests are frequent.
Is there a standard prompt format?
No universal standard exists. Anthropic, OpenAI, and Meta each have their own chat templates. Use the provider-recommended format.
Where to go next