Building an AI agent from scratch in 2026 is the fastest way to understand why every framework exists — and why you might not need one. An agent is not magic: it is a while loop, a model, a list of tools, and a memory strategy. Once you have built the raw version, every abstraction makes sense.
What changed in 2026
- Tool calling is a first-class API feature on all major providers. Structured function schemas are standard; you no longer parse JSON out of freeform text.
- Reasoning models handle multi-step planning better. Models like Claude 3.7 and GPT-4o think before they act, which reduces tool-call errors significantly.
- Streaming agent outputs is expected. Users see intermediate steps; nobody waits for a silent 30-second black box.
- Cost-per-task is a real metric. Teams now budget agents the way they budget CI jobs — per run, with hard caps.
The minimal agent loop
An agent has three moving parts: a model, a set of tools, and a loop that connects them.
import anthropic, json
client = anthropic.Anthropic()
tools = [
{
"name": "search_web",
"description": "Search the web and return top results.",
"input_schema": {
"type": "object",
"properties": {"query": {"type": "string"}},
"required": ["query"],
},
}
]
def run_agent(goal: str, max_steps: int = 10) -> str:
messages = [{"role": "user", "content": goal}]
for _ in range(max_steps):
resp = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
tools=tools,
messages=messages,
)
if resp.stop_reason == "end_turn":
return resp.content[0].text
for block in resp.content:
if block.type == "tool_use":
result = dispatch_tool(block.name, block.input)
messages.append({"role": "assistant", "content": resp.content})
messages.append({
"role": "user",
"content": [{"type": "tool_result",
"tool_use_id": block.id,
"content": result}],
})
return "Max steps reached."
dispatch_tool is just a dict lookup: {"search_web": search_web_fn}[name](**inputs). That's the entire agent pattern.
What changed in 2026
Tool schemas are now validated server-side. Pass an invalid input and you get a structured error back — no silent hallucinated calls.
Memory strategies
| Type |
Implementation |
Cost |
| In-context (short-term) |
Append to messages list |
O(tokens) |
| Summarised context |
Compress old turns with a cheap model |
Low |
| External retrieval (RAG) |
Vector store lookup per step |
Medium |
| Persistent key-value |
Read/write tool backed by a DB |
Medium |
For most agents under 20 turns, in-context is fine. Beyond that, summarise or retrieve.
How to start
- Write the goal as a system prompt. Be specific: "You are an agent that answers questions about our codebase. You have one tool: search_code."
- Add exactly one tool first. Validate the whole loop works before adding a second.
- Log every message. You cannot debug an agent loop you cannot inspect.
- Write a success criterion before running. "The agent returns a PR diff in ≤ 8 steps" is measurable.
- Add a cost guard. Track tokens per step; abort if cumulative cost exceeds a threshold.
Common mistakes
No step cap. Without max_steps, a confused agent loops indefinitely. Always set it.
Too many tools. Giving the model 15 tools on first run causes hesitation and wrong selections. Add tools one at a time as failures demand them.
Mutable global state. Tools that have side effects without logging make debugging impossible. Log inputs and outputs of every tool call.
Trusting the model to know when to stop. Always check stop_reason == "end_turn" explicitly — do not rely on the model voluntarily finishing.
Not handling tool errors. If a tool throws, feed a tool_result with the error message back to the model so it can recover.
What to skip
- Multi-agent orchestration for problems a single agent with 3 tools solves. The coordination overhead rarely pays.
- Streaming frameworks until your non-streaming agent works reliably.
- Fine-tuned "agent models" unless you have measured that a general model fails at your task — most don't need it.
FAQ
Do I need LangChain or LlamaIndex to build an agent?
No. The SDK tool-calling API is sufficient. Frameworks add convenience; they also add complexity. Build raw first.
How do I evaluate my agent?
Define a task set with known correct outcomes. Run the agent and measure resolved-task rate, steps-per-task, and cost-per-task.
What is the right model to use?
Start with a balanced model (Claude Sonnet, GPT-4o). Switch to a cheaper model only once you have evals that prove quality holds.
How do I handle parallel tool calls?
Most frontier models can emit multiple tool-use blocks in one response. Handle them in a list and send all results back before continuing.
Where to go next