Structured Output That Holds in Production: Schema Design, Validation, and Repair

Extraction pipelines are the least glamorous LLM feature and the one most likely to be quietly wrong. Chat output gets read by a human who notices when it is nonsense. A parsed object gets written to a database.

The good news is that the formatting problem is largely solved. Tool calling and constrained decoding — with_structured_output over a Pydantic model, JSON-schema-constrained sampling on the provider side — mean you rarely see a JSONDecodeError anymore. The bad news is that teams read that as "structured output works now" and stop building safeguards. Schema-valid and correct are different properties. A model that must return {"invoice_total": float} will return a float. It will not tell you that it took the subtotal.

This is about the gap between those two properties, and what to put in it.

Failure modes that survive a valid schema

From audits of extraction and classification pipelines, the recurring ones:

Confident fabrication in required fields. A required field with no nullable option is an instruction to produce a value. If the document has no invoice number, the model invents a plausible one. Requiredness is not free.

Enum collapse. Given a category enum of twelve values, models over-select the first, the most generic, and the ones whose names best match surface wording. The distribution of outputs drifts from the distribution of reality, and nothing in the schema notices.

Unit and format drift. amount: float accepts 1200, 1200.00, and 12.00 equally. Dates as strings accept 03/04/2026 in both readings. Currency, timezone, and scale live outside the type system unless you put them in.

Silent truncation of lists. Ask for line_items: list[LineItem] on a long document and you often get the first several, stopping cleanly and validly. Nothing in the object says it is partial.

Cross-field incoherence. Each field passes on its own; together they contradict. Totals that do not sum, end_date before start_date, a status of resolved with a null resolution.

Over-nesting fragility. Deeply nested schemas with optional branches are where quality degrades fastest. The model is holding a tree in its head while reading a document.

Design the schema for the model, not for your database

The schema is a prompt. It is usually the most influential part of the prompt, and it is the part teams write in a hurry by reflecting their ORM.

• Make absence expressible. Every field the source may genuinely lack should be Optional, and the description should say when to use null: "Invoice number exactly as printed. Null if the document shows none — do not infer from the filename." This one change removes most fabrication in our experience.
• Use field descriptions as instructions. Field(description=...) reaches the model. Put the decision rule there, next to the field it governs, rather than in a paragraph at the top of the system prompt that has to be mentally re-associated with every field.
• Encode units in names or types. total_amount_cents: int and currency: Literal["USD", "EUR", ...] beat total: float. effective_date: date in ISO-8601, stated in the description, beats a free string.
• Keep enums small and mutually exclusive, and include an escape hatch. A twelve-value enum with an other member and a required other_description produces better data than twelve values that force a guess. Then count the other rate — it is a free signal about your taxonomy.
• Flatten where you can. Two sequential extractions over a flat schema usually beat one pass over a nested one. Cheaper to evaluate, too, because you can score the stages separately.
• Ask for evidence on fields that matter. For each high-stakes field, request a source_quote alongside the value: the verbatim span the value came from. Then check the quote appears in the input. This turns a grounding claim into a mechanical test — cheap, no judge model, and it catches fabrication that no type check would.

Validate past the type check

Schema validation tells you the shape is right. Add a second layer that asks whether the content is plausible, and run it inline before anything downstream:

1. Groundedness. Does each source_quote literally appear in the source text (after whitespace normalization)? Does each extracted number appear somewhere in the document? Ungrounded numerics are the highest-yield check in most invoice and report pipelines.
2. Arithmetic and temporal coherence. Line items sum to the total within tolerance. Dates ordered correctly. Percentages in range.
3. Completeness heuristics. If the document has twenty rows matching your row pattern and you extracted eight, flag it. A crude regex row count is enough to catch truncation.
4. Referential checks. IDs, SKUs, and account numbers resolved against systems you already own. If it does not exist in your database, the model did not read it.

Each check produces a reason code, not just a boolean. Reason codes are what let you route, measure, and later fix the specific failure.

Repair narrowly, and never in a blind loop

The common reflex on validation failure is to retry the whole extraction, sometimes several times, and accept whatever passes. That hides your error rate and inflates cost and latency for the least reliable inputs.

Better:

• Feed the error back once, scoped to the failing field. Send the specific validation message and ask for that field alone, not a whole re-extraction. Targeted repair converges more often and costs a fraction as much.
• Cap at one repair attempt. If the second pass fails, stop. Beyond one attempt, the signal you are getting is "this input is hard," and more sampling mostly buys you a plausible-looking wrong answer.
• Make partial success first-class. Return the fields that validated, mark the ones that did not as unresolved with their reason codes, and let the consumer decide. An object with three trustworthy fields and one flagged is more useful than four fields of unknown quality.
• Route rather than discard. Unresolved high-stakes fields go to a review queue — in LangGraph, an interrupt before the commit node, keyed by reason code so reviewers see the specific question rather than the whole document.
• Count repairs as a quality metric. Repair rate by field and by document type is your early-warning system. It moves when a provider updates a model, when an upstream OCR step changes, when a new customer sends a new template. Alert on it.

One more thing worth measuring before you tune prompts: how much of your error rate is extraction and how much is the text you handed the model. On scanned and PDF-heavy corpora, a meaningful share of "extraction errors" are parsing errors upstream — collapsed table columns, dropped decimal points. Check the input text for your failures before rewriting the schema.

Evaluate field by field

Document-level accuracy — "was the whole object right" — is the wrong headline metric. It hides which field is broken and it drops as you add fields, so improvements look like regressions.

Build the eval set the way you would any other: real documents, drawn from real traffic, including the templates that caused incidents. Fifty is enough to start. Label the correct value per field, then report per-field precision and recall, with null as a scored answer. Precision on a field measures fabrication; recall measures misses; and scoring null properly is what keeps a model that hedges everything from looking accurate.

Stratify by document type or source. Aggregate numbers hide the one vendor template that fails 40% of the time, and that template is usually where the business impact is.

Then wire it into CI, same as any eval suite, and read the per-example diff on every schema edit. Schema changes have non-local effects: adding a field changes how neighboring fields are filled, tightening one description shifts the enum distribution. Without a diff you will not see it.

What this looks like when it is working

A production extraction pipeline that we would sign off on has: a schema where absence is expressible and units are explicit, evidence spans on the fields that matter, a validation layer with reason codes running inline, one scoped repair attempt, a review queue for what remains, and per-field precision and recall in CI with repair rate on a dashboard.

None of it is exotic. It is the difference between an extractor you can put in front of a ledger and one you can put in a demo.

If you are running extraction at volume and cannot currently state your per-field precision, that is the place to start — and it is a conversation we are happy to have.