> ## Documentation Index
> Fetch the complete documentation index at: https://smithers.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Structured Output and Typed Contracts

> How Smithers turns agent responses into schema-validated rows a program — and a durable engine — can rely on: Zod contracts on tasks, validation-and-repair loops, and OpenAPI-derived tools.

An agent that "did the work" but returned prose is useless to the next step. A review task that answers *"Looks good overall, though the auth change worries me a bit"* cannot gate a deploy: there is no `approved: boolean` to branch on. So orchestration code grows regexes over model text, the regexes break on the next model update, and the pipeline fails at 3am on a response that was *almost* JSON — a markdown fence around it, a trailing comma inside it, a paragraph of apology before it.

The deeper problem is architectural. A durable workflow engine can only resume, retry, score, or replay a step if the step's result is *data with a known shape*. If step outputs are free text, then "resume after crash" means re-reading a transcript and guessing; "replay from step 3" means nothing, because step 3's contribution was never captured as a value. Typed boundaries between tasks are not a convenience — they are the precondition for durability. This is why Smithers makes the output schema part of the task definition itself, not a parsing afterthought.

This page covers how the industry gets structure out of models, how Smithers does it (schema-constrained task outputs, the validation-and-repair loop, durable output rows), and the reverse direction: [OpenAPI-derived tools](/concepts/openapi-tools) that give agents typed inputs into external systems.

## How the industry solves it

Two families: **constrain the decoder** so invalid output is impossible, or **validate-and-retry** after generation. Most production systems now mix both.

| System                                                                                          | Mechanism                                                                                                                                                                   | Tradeoff                                                                                                                                                                 |
| ----------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| [OpenAI Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs)         | `response_format: json_schema` with `strict: true`; the server compiles the schema to a grammar and masks invalid tokens during sampling                                    | Guaranteed schema adherence, but a restricted JSON Schema subset (all fields required, `additionalProperties: false`); first request per schema pays grammar compilation |
| [Anthropic tool use](https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview)     | Define a "tool" whose input schema *is* your output type and force `tool_choice`; the model's tool-call arguments are your structured result                                | The classic pattern before dedicated APIs; arguments are strongly steered but historically not grammar-guaranteed, so callers still validate                             |
| [Outlines](https://github.com/dottxt-ai/outlines)                                               | Compiles regex/JSON Schema to a finite-state machine over the tokenizer vocabulary; masks logits each step                                                                  | Zero invalid outputs on open-weight models; FSM compilation cost per schema; local/serving-framework only                                                                |
| [llguidance](https://github.com/guidance-ai/llguidance)                                         | On-the-fly token-mask computation for full context-free grammars (Lark syntax, embedded JSON Schema), \~50µs/token                                                          | The current speed benchmark for constrained decoding; used inside serving stacks, not a hosted API                                                                       |
| [llama.cpp GBNF](https://github.com/ggml-org/llama.cpp/blob/master/grammars/README.md)          | Hand-written BNF grammars constrain sampling directly                                                                                                                       | Total control, works fully offline; grammars are manual and easy to get subtly wrong                                                                                     |
| [Instructor](https://python.useinstructor.com/) + Pydantic                                      | Free generation → Pydantic validation → on failure, re-prompt with the validation errors (`max_retries`)                                                                    | No provider support needed, validators can encode *semantic* rules; costs extra round-trips on failure                                                                   |
| [Vercel AI SDK `generateObject`](https://ai-sdk.dev/docs/reference/ai-sdk-core/generate-object) | Zod schema in, typed object out; uses the provider's native structured-output channel when available, JSON-mode/prompting otherwise                                         | Clean TS ergonomics; throws `NoObjectGeneratedError` and leaves retry policy to you                                                                                      |
| [BAML](https://boundaryml.com/)                                                                 | Schema-Aligned Parsing: let the model speak freely, then run an error-tolerant parser that coerces near-miss output (fences, trailing commas, wrong quotes) into the schema | Deliberately rejects constrained decoding to preserve reasoning quality; parsing is heuristic, so pathological outputs still fail                                        |

The tradeoff BAML is pointing at is real and studied: [Tam et al., *Let Me Speak Freely?* (EMNLP 2024)](https://arxiv.org/abs/2408.02442) measured significant reasoning degradation when models are forced into strict formats at generation time, with tighter constraints degrading more. Constrained decoding guarantees *shape*, not *quality* — a grammar-perfect wrong answer is still wrong. The pragmatic synthesis most systems have landed on, and the one Smithers implements: let the model work freely (tools, scratch reasoning, long CLI sessions), demand structure only at the end, validate hard, and repair with feedback when validation fails.

## How Smithers does it

### Schemas are the workflow's type system

Output contracts are declared once, at workflow creation, as Zod schemas. `createSmithers` registers each schema as a durable output relation and returns typed references:

```tsx theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { ClaudeCodeAgent, createSmithers, Sequence, Task } from "smithers-orchestrator";
import { z } from "zod";

const { Workflow, smithers, outputs } = createSmithers({
  review: z.object({
    approved: z.boolean(),
    comments: z.string(),
    severity: z.enum(["low", "medium", "high"]),
  }),
  fix: z.object({ summary: z.string(), filesChanged: z.array(z.string()) }),
});

const claude = new ClaudeCodeAgent({ model: "claude-fable-5" });

export default smithers((ctx) => (
  <Workflow name="review-and-fix">
    <Sequence>
      <Task id="review" agent={claude} output={outputs.review}>
        Review the diff in this repo for correctness and security.
      </Task>
      <Task id="fix" agent={claude} output={outputs.fix} retries={2}
            deps={{ review: outputs.review }}>
        {({ review }) =>
          review.approved
            ? "No changes needed; summarize why."
            : `Fix the issues: ${review.comments}`}
      </Task>
    </Sequence>
  </Workflow>
));
```

`outputs.review` is a typed reference, so `output={outputs.reveiw}` is a compile-time error, and the `deps` callback receives `review` already parsed and typed. Downstream reads (`ctx.outputMaybe`, `ctx.output`, `ctx.outputRows`) return rows that already passed validation — consuming code never sees unvalidated model text. Rows are keyed `(runId, nodeId, iteration)` and persisted to per-schema tables; `runId`, `nodeId`, and `iteration` are stripped from agent responses before validation and auto-populated on write, so the model cannot forge row identity.

Tasks have [three modes](/how-it-works#tasks-three-modes) — agent, compute, static — and all three write to the same typed relations, so a hand-computed value and an LLM answer are interchangeable to consumers.

### Getting structure out of the model

Smithers picks the extraction strategy per agent:

* **Native structured output.** Agents that declare `supportsNativeStructuredOutput = true` (`AnthropicAgent`, `OpenAIAgent` by default) get the Zod schema forwarded through the AI SDK's `Output.object({ schema })` channel — the provider-side constrained/steered path from the table above. `CodexAgent` passes `--output-schema` to the Codex CLI.
* **Prompt-based extraction.** CLI harness agents (including `ClaudeCodeAgent`) work freely for the whole session; Smithers injects JSON instructions into the prompt, then extracts the *last* balanced JSON object from the response — searching from the end so an intermediate tool result doesn't shadow the final answer — with markdown-fence handling. This is the free-form-then-parse side of the Tam et al. tradeoff, on purpose: the agent's multi-hour coding session is never token-constrained; only the final report is.

<Note>Local OpenAI-compatible servers (e.g. llama.cpp behind an OpenAI shim) often advertise but don't fully implement JSON-schema structured output. Smithers detects this failure class and suggests `new OpenAIAgent({ nativeStructuredOutput: false })` to fall back to prompt-based extraction.</Note>

### Validation failure is a repair loop, not a crash

Extraction and validation failures each get an in-attempt correction budget of `maxSchemaRetries` (default 3, settable per task). Two correction kinds, both visible in logs as `engine:schema-retry`:

1. **No JSON found** — the agent is re-asked for the structured result. CLI agents *resume their own session* (`claude --resume`, `codex exec resume`), so the correction has the full task context; agents without a session get the original task text replayed into the repair prompt, because a context-free repair otherwise produces schema-valid but amnesiac values.
2. **JSON found, schema mismatch** — the Zod issues are formatted (`- path: message`) and sent back Instructor-style with the schema shape restated; native-structured-output agents get a structured retry, SDK agents get the recorded conversation replayed with the correction appended.

Only when the in-attempt budget is exhausted does the attempt fail — and then task-level `retries` takes over with a fresh attempt. `retries={2}` means 3 attempts × (1 generation + up to 3 repairs) = up to 12 chances at a valid row. Every correction is journaled: `schemaRetryAttempts` lands in the error details and attempt metadata, so you can see exactly how hard a row was to obtain.

### Typed rows are what durability stands on

Because every completed task is a Zod-validated row in a known table, the engine can:

* **Resume** a crashed run by re-rendering the workflow against persisted rows — completed tasks simply *have* output, so they don't re-execute ([Durability](/capabilities/durability)).
* **Score** outputs: [scorers and eval suites](/capabilities/evals-and-optimization) receive parsed objects, not transcripts.
* **Replay and fork** from a checkpoint ([Time Travel](/capabilities/time-travel)): a frame is meaningful because the state at that frame is typed rows, not conversation history.

### Typed inputs: OpenAPI-derived tools

The same discipline runs in the other direction. `createOpenApiTools` compiles an OpenAPI 3.x spec into AI SDK tools — one per operation, Zod schemas converted from the spec's JSON schemas — so the agent's *calls into your systems* are typed too. Curate the surface with `include`/`exclude`/`operations`, and generate a reusable module from the CLI:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
bunx smithers-orchestrator openapi list ./api/openapi.yaml       # audit what an agent could call
bunx smithers-orchestrator openapi generate ./api/openapi.yaml ./src/openapi-tools.js
```

See [OpenAPI Tools](/concepts/openapi-tools) for auth, redirect-origin protection, response-size caps, and current limitations (JSON bodies only, no cookie params, OpenAPI 3.x only).

## Guarantees and limits

* **Guaranteed:** a persisted output row always matches its Zod schema. There is no code path that writes an invalid row; consumers can trust `row.approved` exists and is boolean.
* **Not guaranteed:** semantic correctness. Schema validation catches shape, not lies — pair it with [scorers](/capabilities/evals-and-optimization) for quality.
* **Outputs must be JSON objects.** Plain text is rejected with a pointed error; wrap it as `z.object({ text: z.string() })` and read `deps.<task>.text`.
* **Repair costs real tokens.** Each correction is another model call. A schema the model chronically misses (deeply nested, ambiguous field names) burns budget — flatten it.
* **Constrained decoding only where the provider supports it;** everywhere else it's extraction + repair, which is robust but probabilistic. The 12-chances arithmetic above is a budget, not a proof.
* **Enum/format edge cases** in provider-native structured output vary by vendor; Smithers surfaces the provider error with a compatibility hint rather than papering over it.

## Operating it

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
smithers output <runId> <nodeId> --pretty      # print a node's validated output row (--iteration N inside loops)
smithers node <runId> <nodeId>                 # attempts, schema retries, tool calls
smithers logs <runId>                          # engine:schema-retry entries show each correction and its Zod issues
smithers graph workflow.tsx                    # inspect the typed dependency graph without executing
smithers eval workflow.tsx suite.jsonl         # run the contract against an eval suite
```

MCP equivalents for agents driving Smithers: `get_node_detail`, `get_run`, `list_artifacts`.

## See also

* [How It Works — Tasks: three modes](/how-it-works#tasks-three-modes) — the canonical task/output reference
* [OpenAPI Tools](/concepts/openapi-tools) — typed agent-to-API surface
* [Durability](/capabilities/durability) — what typed rows buy you when the process dies
* [Evals and Optimization](/capabilities/evals-and-optimization) — scoring beyond schema validity