Structured Output

Sooner or later something downstream needs JSON rather than prose: a triage classification, a set of extracted fields, a decision object your code acts on. Asking politely in the prompt gets you valid JSON most of the time, and "most of the time" is an incident at Sundry's volume — a 2% parse failure across 4,200 tickets a week is 84 tickets that fell into an exception handler.

There are three levels of enforcement available, they remove different classes of failure, and they cost different things. The one mistake that spans all three is assuming the strongest of them removes the need for validation in your own code. It does not, and the reason is the whole point of this page.

Three Levels of Enforcement

The levels are cumulative in strength and each one buys a narrower guarantee than teams assume.

Prompt-level asking is where everyone starts and where nobody should finish. It fails in specific, recognizable ways: the model wraps the object in a code fence, prefaces it with "Here's the classification:", emits two objects when the ticket had two intents, or produces a trailing comma. Provider-side schema validation removes that entire class. Constrained decoding removes it at the token level — invalid tokens are masked out of the distribution before sampling, so malformed output is not merely unlikely, it is unreachable. It requires control over decoding, which means a self-hosted model or a library that owns the sampling loop, and it can nudge the model's wording where the grammar is tight.

Schemas Are Prompts

A schema is not just a validator. It goes into the context, the model reads it, and every name, description and enum in it is doing prompt work. This is the fact on this page that pays back fastest: a field called reason_code with six enumerated values outperforms a free-text field called reason on accuracy and on everything downstream of it, because the model is choosing from a list instead of composing a phrase, and your code is switching on a constant instead of matching strings.

{
  "name": "refund_decision",
  "schema": {
    "type": "object",
    "properties": {
      "amount_cents": {"type": "integer", "minimum": 0, "maximum": 15000},
      "source":       {"type": "string", "enum": ["seller_balance", "sundry_account"]},
      "policy_ref":   {"type": "string",
                       "description": "Id of the policy document this decision cites"},
      "needs_human":  {"type": "boolean",
                       "description": "True when the amount exceeds the $150 ceiling"}
    },
    "required": ["amount_cents", "source", "policy_ref", "needs_human"],
    "additionalProperties": false
  }
}

Four fields, flat, named in the language of the business. The amount is in cents as an integer, so there is no float to round wrong, and its maximum encodes the $150 ceiling in the schema itself — the model is told the boundary in the same breath as the field. The source is an enum of two, because a marketplace refund comes out of a seller balance or out of Sundry's own account and there is no third answer. The policy reference forces the model to name the document it relied on, which is what makes the decision auditable later. And additionalProperties is false, so a model that invents a helpful extra field is rejected rather than silently ignored.

Write schemas for a reader who cannot ask you a question, because that is exactly what the model is. Descriptions earn their tokens; a field called source with no description is a coin flip, and the same field with one sentence explaining what the two values mean is not. Chapter 3 makes the identical argument about tool schemas, for the identical reason.

Validate Anyway

Every level above enforces shape. None of them enforces truth. A refund decision can be perfectly schema-valid — integer amount inside the ceiling, a legal enum value, a plausible-looking policy id, the boolean set — and still be wrong in every way that matters, because the order id does not exist, the policy reference names a document that was never retrieved, or the seller runs the statutory 14-day window rather than Sundry's 30.

So validate on receipt, in your own code, against your own systems. Does that order exist and belong to this buyer? Does the cited policy document appear in what was actually retrieved this run? Is the amount consistent with the charges on the order? Those checks cannot live in a schema and must not live in the prompt, because a prompt is a request and a check is a guarantee. This is the same boundary the whole book keeps returning to: the model proposes, your code disposes, and the enforcement always sits outside the model.

Structured Output vs Tool Calls

A tool call is already structured output with a function name attached. The model emits a name plus an argument object validated against the tool's schema, which is exactly what a structured response is, plus a dispatch target. That similarity leads teams into a specific mistake: defining a tool the model is meant to "call" purely to get a shaped object back, with no implementation behind it.

The rule is simple. If your code will execute something, it is a tool. If you only need data, ask for a schema-constrained response instead. The fake tool costs a schema in every request for the rest of the ticket, it sits in the tool list competing for selection against the nine real ones, and it makes the model's job harder for no benefit — Chapter 3 shows how quickly selection accuracy falls as a tool surface fills with near-duplicates. One schema-constrained call is cheaper, clearer in the trace, and impossible to select by accident.

Failure Handling

When an object fails validation, repair once. Send the invalid object back with the validator's actual error message attached — "policy_ref: SU-POL-88 was not among the documents retrieved on this run" — and ask for a corrected object. A specific error is information the model can act on, and one repair attempt fixes the large majority of real failures.

A second attempt rarely helps and a third is a loop. If the repair fails, stop: fall into a defined path, which at Sundry means escalating the ticket to a person with both attempts attached. Count repairs as a metric, because a rising repair rate is one of the earliest signals that a prompt, a schema or a model version has drifted — Chapter 9 treats it as a leading indicator rather than noise.

Where Sundry Uses It

Three places, and they are worth distinguishing because they have different tolerances. The triage classifier runs once per ticket and returns a category and a confidence, driving which queue and which tool subset the ticket gets; a wrong classification is recoverable, so this one runs at the schema level and is checked against the enum. The refund decision object above is the strictest: money moves on it, so it gets the strongest enforcement the provider offers plus every semantic check listed earlier.

The third is the evaluation judge in Chapter 9, which grades a run and returns a verdict with a reason code and a citation of the evidence it used. That one matters because the whole eval suite reads its output as data — a judge whose verdicts sometimes fail to parse produces a resolution figure that quietly excludes its own hardest cases, which is a measurement error dressed as a parsing bug.