Code · June 9, 2026

Structured outputs guide for 2026: JSON mode, tool schemas, and Pydantic

Q: Can I use structured outputs with streaming?

Yes — OpenAI's `stream=True` with `response_format` works; parse the completed stream before Pydantic validation.

Q: How do I handle optional fields that the model keeps omitting?

Make them `Optional` in the Pydantic model with a `None` default, and add a description asking the model to include them when available.

Getting reliable structured data from LLMs in 2026 is solved with the right API features. This guide covers JSON mode, tool schemas, and validation patterns.

By ByteLedger Team

Getting structured data out of a language model reliably was a significant pain point in 2023–2024. In 2026 it is a solved problem at the API level. JSON mode, tool-calling schemas, and provider-specific structured output features give you machine-readable output without brittle regex or prompt-wrestling.

What changed in 2026

Tool-calling is universal across Anthropic, OpenAI, Google, and Mistral — one pattern works everywhere.
JSON schema constraints are server-enforced on OpenAI's response_format and Anthropic's tool-choice API. The model cannot produce output that violates the schema.
Pydantic v2 became the standard Python integration layer — .model_json_schema() generates the schema, .model_validate() validates the response.
Structured output tokens are cheaper on most providers because schema-constrained generation is more efficient.

Three options, ranked by reliability

Method	Reliability	Flexibility	Complexity
Tool-calling (forced)	Highest	Schema-constrained	Low
JSON schema `response_format`	High	Schema-constrained	Low
JSON mode (no schema)	Medium	Any valid JSON	Low
Prompt-instructed JSON	Low	Anything	Low (but fragile)

Always prefer the top options. Drop to lower options only when the provider does not support schemas.

Tool-calling for structured output (recommended)

The most reliable pattern: define the output structure as a tool schema, force the model to call that tool.

import anthropic
from pydantic import BaseModel

class InvoiceExtraction(BaseModel):
    vendor_name: str
    invoice_number: str
    total_amount: float
    currency: str
    due_date: str | None = None

client = anthropic.Anthropic()

def extract_invoice(text: str) -> InvoiceExtraction:
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=512,
        tools=[{
            "name": "extract_invoice",
            "description": "Extract structured data from an invoice",
            "input_schema": InvoiceExtraction.model_json_schema(),
        }],
        tool_choice={"type": "tool", "name": "extract_invoice"},
        messages=[{"role": "user", "content": f"Extract invoice data:\n{text}"}],
    )
    tool_input = response.content[0].input
    return InvoiceExtraction.model_validate(tool_input)

The tool_choice forces the model to produce only the tool call — no prose wrapping the JSON.

OpenAI JSON schema `response_format`

from openai import OpenAI
from pydantic import BaseModel

class SupportTicket(BaseModel):
    category: str
    priority: str  # "low" | "medium" | "high"
    summary: str

client = OpenAI()

def classify_ticket(text: str) -> SupportTicket:
    response = client.beta.chat.completions.parse(
        model="gpt-4o-2024-11-20",
        response_format=SupportTicket,
        messages=[{"role": "user", "content": text}],
    )
    return response.choices[0].message.parsed

client.beta.chat.completions.parse handles schema generation and parsing automatically with Pydantic models.

JSON mode (lightweight)

When you need valid JSON but the schema varies dynamically:

response = client.chat.completions.create(
    model="gpt-4o",
    response_format={"type": "json_object"},
    messages=[{
        "role": "user",
        "content": "Return a JSON object with keys: name, score, reason."
    }],
)
data = json.loads(response.choices[0].message.content)

JSON mode guarantees valid JSON syntax but does not enforce key names or types — validate manually.

Validation and error handling

Even with server-enforced schemas, add a validation layer:

import json
from pydantic import ValidationError

def safe_extract(text: str) -> InvoiceExtraction | None:
    try:
        return extract_invoice(text)
    except ValidationError as e:
        logging.error("Schema validation failed: %s", e)
        return None
    except anthropic.BadRequestError as e:
        # Happens when input is too short or nonsensical for extraction
        logging.warning("Model refused structured output: %s", e)
        return None

Handling enum and union types

Define enums explicitly in the schema to constrain categorical fields:

from enum import Enum
from pydantic import BaseModel

class Priority(str, Enum):
    low = "low"
    medium = "medium"
    high = "high"

class Ticket(BaseModel):
    category: str
    priority: Priority  # Model can only output "low", "medium", or "high"
    summary: str

Pydantic's .model_json_schema() generates the enum constraint automatically.

How to pick

Known output schema? Use tool-calling (Anthropic) or response_format with Pydantic (OpenAI).
Schema varies at runtime? Build the JSON schema programmatically and pass it directly.
Just need valid JSON, any structure? JSON mode is fine.
No structured output API available? Use few-shot examples of the exact output format + manual parsing. Avoid this path where possible.

Common mistakes

Not forcing tool use. Specifying a tool without tool_choice: forced/specific lets the model decide to call it or not — it often does not.

Nested schemas without depth limits. Deeply nested Pydantic models with many optional fields confuse models into omitting required keys. Flatten where possible.

Skipping validation after parsing. Trust but verify — even server-constrained output can fail Pydantic validators if you have custom validators or cross-field rules.

Large lists without count hints. Asking for "a list of all items" can produce truncated results. Specify max_items in the description or add a count field to prompt completeness.

What to skip

Regex extraction from LLM output — it is fragile, model-version-sensitive, and entirely unnecessary with modern APIs.
Multi-round extraction loops where you ask the model to fix its own JSON — use server-enforced schemas and it will not break in the first place.

FAQ

Is tool-calling schema enforcement 100% reliable? Near-100% on modern frontier models. Rare failures occur on extremely complex nested schemas. Validate defensively regardless.

Can I use structured outputs with streaming? Yes — OpenAI's stream=True with response_format works; parse the completed stream before Pydantic validation.

What about open-source models? Llama 4, Mistral Large 2, and Phi-4 all support function calling schemas. Quality drops for complex schemas compared to frontier models.

How do I handle optional fields that the model keeps omitting? Make them Optional in the Pydantic model with a None default, and add a description asking the model to include them when available.