Structured Outputs with JSON Schema in Production

Key facts, with sources

• At DevDay 2025 OpenAI reported 800 million weekly ChatGPT users, 4 million developers building on its platform, and roughly 8 billion API tokens processed per minute. (CNBC)
• ChatGPT reached 900 million weekly active users by late February 2026, up from 800 million at DevDay in October 2025. (TechCrunch)
• By March 2026 OpenAI's APIs were processing more than 15 billion tokens per minute, roughly doubling from the rate reported at DevDay 2025. (Panto AI OpenAI Statistics)
• OpenAI's published API pricing discounts cached input tokens by 90 percent on supported GPT models, which materially cuts costs for agents that resend long system prompts. (OpenAI API Pricing Docs)
• OpenAI raised $122 billion in new funding in 2026 to accelerate the next phase of AI development, one of the largest private raises in history. (OpenAI)

JSON mode versus Structured Outputs: not the same guarantee

JSON mode promises only that the response is valid JSON — the model still chooses the keys, nesting, and types, so you get parseable output with missing fields, renamed keys, or a string where you needed an integer. Structured Outputs is a categorically stronger contract: you supply a JSON Schema, and constrained decoding makes it impossible for the model to emit tokens that violate it. The output parses and conforms, every time.

Before this existed, production extraction code was a loop: generate, attempt parse, retry with the error appended, give up after three attempts. That pattern — still present in older codebases I audit — burns tokens, adds latency, and fails anyway under load. If any code in your system still retries on parse errors against a model that supports Structured Outputs, deleting that loop is a free reliability upgrade.

Define schemas as Pydantic models

The Python SDK accepts a Pydantic model directly and returns a parsed, typed instance — one schema definition serves the API contract, runtime validation, and your type checker. Nested models, lists, enums, and nullable fields all translate cleanly.

import os
from pydantic import BaseModel
from openai import OpenAI

MODEL = os.environ["OPENAI_MODEL"]  # set to the latest model id
client = OpenAI()

class LineItem(BaseModel):
    description: str
    amount_cents: int

class Invoice(BaseModel):
    vendor: str
    invoice_date: str | None  # strict mode: optionals are nullable, not omitted
    total_cents: int
    line_items: list[LineItem]

completion = client.beta.chat.completions.parse(
    model=MODEL,
    messages=[
        {"role": "system", "content": "Extract the invoice from the user's email."},
        {"role": "user", "content": raw_email_text},
    ],
    response_format=Invoice,
)

msg = completion.choices[0].message
if msg.refusal:
    handle_refusal(msg.refusal)
else:
    invoice: Invoice = msg.parsed  # typed, guaranteed to match the schema

Strict-mode rules that bite in practice

Strict schemas obey rules that surprise teams porting existing models. Every field must be required — optionality is expressed as a union with null, not by omitting the key, so downstream consumers must expect explicit nulls. Additional properties are disallowed, which is what makes the guarantee airtight but means the model cannot volunteer extra fields. And only a subset of JSON Schema keywords participates in enforcement; treat numeric ranges and string patterns as documentation and enforce them yourself after parsing.

Two operational notes. The first request with a new schema can carry extra latency while the schema is processed, with subsequent calls fast — warm new schemas before traffic hits them. And deeply nested or enormous schemas eventually hit complexity limits; if a schema is approaching them, that is usually a sign the extraction should be decomposed into multiple focused calls anyway.

Schema-valid is not correct

Structured Outputs guarantees shape, not truth. The model can emit a perfectly conformant invoice with the total off by a factor of a hundred because it misread a currency format, or a plausible date that appears nowhere in the source. Constrained decoding eliminates parsing failures; it does nothing about extraction errors — and by making outputs look clean, it can lull teams into skipping the validation that catches them.

So keep a post-parse validation layer for business rules: totals should reconcile with line items, dates should fall in plausible ranges, and values that matter should be cross-checkable against the source text. Route validation failures to a retry with feedback or a human queue, and maintain a small accuracy eval — documents with known-correct extractions — so you measure field-level accuracy, not just parse success, across prompt and model changes.

Handle refusals and evolve schemas like API contracts

When a request trips safety behavior, the model returns an explicit refusal instead of schema-conforming output, surfaced as a distinct field on the message. Check it before touching parsed data and design a real code path for it — log, fall back, or surface a message — because an unhandled refusal otherwise becomes a confusing null-pointer crash far from its cause.

Treat the schema itself as a versioned API contract, because that is what it is: producers (prompts and models) and consumers (your downstream code) both depend on it. Additive changes — a new nullable field — are safe; renames and type changes are breaking and need a migration window with both versions accepted. Embed a schema version field in the output and log it with every response, so when extraction behavior shifts you can tell whether the schema, the prompt, or the model changed.

When to hire senior help

Bring in senior help when you move from a working prototype to production traffic, because cost controls, evals, rate-limit handling, and fallback behavior determine whether the unit economics work. An experienced engineer usually pays for themselves by cutting token spend and preventing outages rather than by writing the first prompt. If your stack includes React Native + Python + AI, a senior engineer who owns the full product beats coordinating multiple juniors.

Bottom line

Dhairya Senjaliya ships AI — OpenAI Development projects worldwide — book a scoping call to discuss your specific situation.

Common pitfalls to avoid

• ✕Hardcoding a single flagship model ID for every call instead of routing by task, paying GPT-5-tier prices for classification work a nano-tier model handles at a fraction of the cost
• ✕Putting volatile content like timestamps and user IDs at the top of prompts, which breaks prefix caching and forfeits the 90 percent cached-input discount
• ✕Building on deprecated surfaces like the legacy Completions or wound-down fine-tuning APIs instead of the current Responses API and agent tooling
• ✕Launching with no spend caps or per-user rate limits, so a retry loop or a single abusive user burns a month's API budget overnight

Frequently asked questions

Sources

• CNBC
• TechCrunch
• Panto AI OpenAI Statistics
• OpenAI API Pricing Docs
• OpenAI