Somewhere in your codebase there is a function that asks an LLM for JSON and then prays. The prompt says "return JSON, no markdown", and the model returns a fenced code block with a sentence before it, a trailing comma, and one key renamed. json.loads() throws, so you retry. The retry returns different JSON with a different key renamed.
That retry loop is a familiar kind of waste. Every attempt costs tokens, latency, and a bit of your sanity. And the fix is not a better prompt.
OpenAI and Anthropic both ship structured outputs. The API compiles your JSON Schema into a grammar and constrains token generation, so the model cannot emit output outside the schema. Valid JSON with the right types and every required key, guaranteed while the tokens are being chosen instead of patched up after the fact. My honest take: prompt-only JSON is fine for demos and nothing else.
Prerequisites
- Python 3.10+
pip install openai anthropic pydantic- API keys:
OPENAI_API_KEYandANTHROPIC_API_KEY - About 10 minutes
Why prompting for JSON is not enough
A language model generates tokens, not data structures. Nothing in the sampling process stops it from writing code fences, adding prose, renaming keys, or inventing enum values. A strong prompt lowers the probability of those failures. It does not remove them, and the failure modes are exactly the ones that hurt:
- Invalid syntax: fences, trailing commas, truncated output
- Missing or renamed required keys
- Wrong types, like
"42"instead of42 - Extra keys your parser ignores or chokes on
Each one costs you a retry, and the retry can fail in a different way. Structured outputs remove the whole category. With constrained decoding, the schema is enforced while the tokens are being sampled, so json.loads stops being a gamble.
OpenAI: response_format and chat.completions.parse
OpenAI has supported structured outputs since GPT-4o. For new projects the docs recommend gpt-5.6. There are two levels: the raw API and the SDK helper.
Raw JSON Schema
The chat completions endpoint takes a response_format with type: "json_schema":
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
model="gpt-5.6",
messages=[{"role": "user", "content": "How do I solve 8x + 7 = -23?"}],
response_format={
"type": "json_schema",
"json_schema": {
"name": "math_reasoning",
"strict": True,
"schema": {
"type": "object",
"properties": {
"steps": {
"type": "array",
"items": {
"type": "object",
"properties": {
"explanation": {"type": "string"},
"output": {"type": "string"},
},
"required": ["explanation", "output"],
"additionalProperties": False,
},
},
"final_answer": {"type": "string"},
},
"required": ["steps", "final_answer"],
"additionalProperties": False,
},
},
},
)
Two things matter here. strict: true turns on schema enforcement, and the schema has to be fully specified: every property listed in required, additionalProperties: false, and nested objects treated the same way. The SDK helpers take care of that for you.
Pydantic, the practical way
Writing raw JSON Schema by hand gets old fast. The Python SDK accepts a Pydantic model directly:
from pydantic import BaseModel
from openai import OpenAI
class CalendarEvent(BaseModel):
name: str
date: str
participants: list[str]
client = OpenAI()
completion = client.chat.completions.parse(
model="gpt-5.6",
messages=[
{"role": "system", "content": "Extract the event information."},
{"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
],
response_format=CalendarEvent,
)
event = completion.choices[0].message.parsed
print(event.name, event.participants)
message.parsed is already a CalendarEvent instance. No json.loads, no validation dance. Two edges are still on you: safety refusals come back in message.refusal, and hitting max_tokens returns an incomplete response. Check both:
message = completion.choices[0].message
if message.refusal:
print("refused:", message.refusal)
elif message.content:
print(message.content)
else:
raise Exception("No response content")
Prefer the Responses API? The same helper exists there: client.responses.parse(model=..., input=..., text_format=CalendarEvent), and the parsed object lands in response.output_parsed.
Anthropic: output_config.format and messages.parse
Anthropic's structured outputs work the same way, with the parameter at output_config.format and type: "json_schema". Supported models include claude-opus-5, claude-sonnet-5, and the claude-haiku-4-5 family.
Raw JSON Schema
from anthropic import Anthropic
client = Anthropic()
response = client.messages.create(
model="claude-opus-5",
max_tokens=1024,
messages=[
{
"role": "user",
"content": (
"Extract the key information from this email: John Smith "
"([email protected]) is interested in our Enterprise plan and "
"wants to schedule a demo for next Tuesday at 2pm."
),
}
],
output_config={
"format": {
"type": "json_schema",
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"email": {"type": "string"},
"plan_interest": {"type": "string"},
"demo_requested": {"type": "boolean"},
},
"required": ["name", "email", "plan_interest", "demo_requested"],
"additionalProperties": False,
},
}
},
)
print(next(block.text for block in response.content if block.type == "text"))
The response is valid JSON in the text content block. Same discipline as OpenAI: required listed explicitly, additionalProperties: false.
Pydantic with messages.parse
The Python SDK's messages.parse takes the model directly. output_format is a convenience parameter that the SDK translates to output_config.format internally:
from pydantic import BaseModel
from anthropic import Anthropic
class ContactInfo(BaseModel):
name: str
email: str
plan_interest: str
demo_requested: bool
client = Anthropic()
response = client.messages.parse(
model="claude-opus-5",
max_tokens=1024,
messages=[
{
"role": "user",
"content": (
"Extract the key information from this email: John Smith "
"([email protected]) is interested in our Enterprise plan and "
"wants to schedule a demo for next Tuesday at 2pm."
),
}
],
output_format=ContactInfo,
)
print(response.parsed_output)
response.parsed_output is a validated ContactInfo instance.
Mini project: an invoice extractor
Enough single-object examples. This is the pattern you will actually use: a nested schema with line items and an enum, driven by one Pydantic model on both providers.
from enum import Enum
from pydantic import BaseModel, Field
class Currency(str, Enum):
USD = "USD"
EUR = "EUR"
IDR = "IDR"
class LineItem(BaseModel):
description: str
quantity: int
unit_price: float
amount: float
class Invoice(BaseModel):
vendor: str = Field(description="Company that issued the invoice")
invoice_number: str
currency: Currency
line_items: list[LineItem]
total: float
due_date: str = Field(description="ISO 8601 date, for example 2026-09-01")
Two notes on the model. Field(description=...) is not decoration: it is the only place the model sees what you mean by due_date or currency, so write it like an instruction. And keep enum values primitive. Both providers restrict enums to strings, numbers, booleans, and null.
The extractors, one per provider, both returning the same Invoice:
def extract_with_openai(text: str) -> Invoice:
from openai import OpenAI
client = OpenAI()
completion = client.chat.completions.parse(
model="gpt-5.6",
messages=[
{
"role": "system",
"content": "Extract invoice data. Copy amounts as written, do not recalculate them.",
},
{"role": "user", "content": text},
],
response_format=Invoice,
)
return completion.choices[0].message.parsed
def extract_with_anthropic(text: str) -> Invoice:
from anthropic import Anthropic
client = Anthropic()
response = client.messages.parse(
model="claude-opus-5",
max_tokens=2048,
messages=[{"role": "user", "content": text}],
output_format=Invoice,
)
return response.parsed_output
Run it:
raw_invoice = """INVOICE INV-2026-0041
PT Maju Bersama, Jakarta
2x Server rack rails @ 450000 = 900000
1x KVM console @ 2800000 = 2800000
Total: IDR 3,700,000
Due date: 2026-09-01"""
invoice = extract_with_openai(raw_invoice)
print(invoice.vendor) # PT Maju Bersama
print(invoice.currency.value) # IDR
print(invoice.total) # 3700000.0
for item in invoice.line_items:
print(f"- {item.description}: {item.amount}")
Swap in extract_with_anthropic and the same model works. That is the whole point of defining the schema once in Pydantic: it stays portable across providers, and the JSON Schema it generates is what each API validates against.
Gotchas that will bite you
The schema is a subset of JSON Schema. Both providers support only part of the spec. Anthropic documents the limits explicitly and returns a 400 error for unsupported features: no numerical constraints like minimum, no string length constraints like minLength, and array minItems only 0 or 1. OpenAI's strict mode demands the same fully specified shape, so keep schemas to primitives, enums, nested objects, and arrays of those. If you need a value between 1 and 100, put that in the description and validate in code.
Recursive schemas. OpenAI supports them: define the model with a self-reference and call model_rebuild(). Anthropic does not support recursive schemas. For nested trees like UI layouts, OpenAI is the easier path.
Enum casing. Anthropic does not guarantee the capitalization of string enum values. A schema with "Conversation topic 3" can come back as "Conversation Topic 3". Compare case-insensitively, and never define enum values that differ only by capitalization.
Grammar compilation latency. Anthropic compiles your schema into a grammar. The first request with a new schema is noticeably slower, compiled grammars are cached for 24 hours from last use, and the cache invalidates when the schema structure or the tool set changes.
Token costs. Anthropic injects a system prompt describing the expected format. It costs tokens like any other system prompt, and changing output_config.format invalidates the prompt cache for that thread.
Refusals and truncation are still your problem. Structured outputs guarantee the shape of the answer, not that the model answered. Handle message.refusal on OpenAI, check stop_reason on Anthropic, and treat responses near max_tokens as incomplete.
When to use structured outputs vs function calling
Both providers draw the same line. If the model has to trigger something in your system, like a database query or a tool call, use function calling. OpenAI exposes it through tools; Anthropic adds strict: true on a tool to validate its input_schema. If the model's reply to the user should itself be structured, a UI payload or an extracted record, use a structured response format.
Plain JSON mode still exists on OpenAI (json_object) for older models. There is no reason to pick it for new work: it guarantees valid JSON and nothing about the shape.
Next steps
- Wrap the extractor in a retry that only fires on refusals and truncation, never on schema violations, because those should no longer happen
- Keep a validation layer anyway. Structured outputs kill most parsing bugs, but a bad source document can still produce a bad extraction
- Run the same Pydantic model through both providers on a small eval set. The schema is identical; the differences show up in edge cases like enum casing