approved: boolean to branch on. So orchestration code grows regexes over model text, the regexes break on the next model update, and the pipeline fails at 3am on a response that was almost JSON — a markdown fence around it, a trailing comma inside it, a paragraph of apology before it.
The deeper problem is architectural. A durable workflow engine can only resume, retry, score, or replay a step if the step’s result is data with a known shape. If step outputs are free text, then “resume after crash” means re-reading a transcript and guessing; “replay from step 3” means nothing, because step 3’s contribution was never captured as a value. Typed boundaries between tasks are not a convenience — they are the precondition for durability. This is why Smithers makes the output schema part of the task definition itself, not a parsing afterthought.
This page covers how the industry gets structure out of models, how Smithers does it (schema-constrained task outputs, the validation-and-repair loop, durable output rows), and the reverse direction: OpenAPI-derived tools that give agents typed inputs into external systems.
How the industry solves it
Two families: constrain the decoder so invalid output is impossible, or validate-and-retry after generation. Most production systems now mix both.
The tradeoff BAML is pointing at is real and studied: Tam et al., Let Me Speak Freely? (EMNLP 2024) measured significant reasoning degradation when models are forced into strict formats at generation time, with tighter constraints degrading more. Constrained decoding guarantees shape, not quality — a grammar-perfect wrong answer is still wrong. The pragmatic synthesis most systems have landed on, and the one Smithers implements: let the model work freely (tools, scratch reasoning, long CLI sessions), demand structure only at the end, validate hard, and repair with feedback when validation fails.
How Smithers does it
Schemas are the workflow’s type system
Output contracts are declared once, at workflow creation, as Zod schemas.createSmithers registers each schema as a durable output relation and returns typed references:
outputs.review is a typed reference, so output={outputs.reveiw} is a compile-time error, and the deps callback receives review already parsed and typed. Downstream reads (ctx.outputMaybe, ctx.output, ctx.outputRows) return rows that already passed validation — consuming code never sees unvalidated model text. Rows are keyed (runId, nodeId, iteration) and persisted to per-schema tables; runId, nodeId, and iteration are stripped from agent responses before validation and auto-populated on write, so the model cannot forge row identity.
Tasks have three modes — agent, compute, static — and all three write to the same typed relations, so a hand-computed value and an LLM answer are interchangeable to consumers.
Getting structure out of the model
Smithers picks the extraction strategy per agent:- Native structured output. Agents that declare
supportsNativeStructuredOutput = true(AnthropicAgent,OpenAIAgentby default) get the Zod schema forwarded through the AI SDK’sOutput.object({ schema })channel — the provider-side constrained/steered path from the table above.CodexAgentpasses--output-schemato the Codex CLI. - Prompt-based extraction. CLI harness agents (including
ClaudeCodeAgent) work freely for the whole session; Smithers injects JSON instructions into the prompt, then extracts the last balanced JSON object from the response — searching from the end so an intermediate tool result doesn’t shadow the final answer — with markdown-fence handling. This is the free-form-then-parse side of the Tam et al. tradeoff, on purpose: the agent’s multi-hour coding session is never token-constrained; only the final report is.
Local OpenAI-compatible servers (e.g. llama.cpp behind an OpenAI shim) often advertise but don’t fully implement JSON-schema structured output. Smithers detects this failure class and suggests
new OpenAIAgent({ nativeStructuredOutput: false }) to fall back to prompt-based extraction.Validation failure is a repair loop, not a crash
Extraction and validation failures each get an in-attempt correction budget ofmaxSchemaRetries (default 3, settable per task). Two correction kinds, both visible in logs as engine:schema-retry:
- No JSON found — the agent is re-asked for the structured result. CLI agents resume their own session (
claude --resume,codex exec resume), so the correction has the full task context; agents without a session get the original task text replayed into the repair prompt, because a context-free repair otherwise produces schema-valid but amnesiac values. - JSON found, schema mismatch — the Zod issues are formatted (
- path: message) and sent back Instructor-style with the schema shape restated; native-structured-output agents get a structured retry, SDK agents get the recorded conversation replayed with the correction appended.
retries takes over with a fresh attempt. retries={2} means 3 attempts × (1 generation + up to 3 repairs) = up to 12 chances at a valid row. Every correction is journaled: schemaRetryAttempts lands in the error details and attempt metadata, so you can see exactly how hard a row was to obtain.
Typed rows are what durability stands on
Because every completed task is a Zod-validated row in a known table, the engine can:- Resume a crashed run by re-rendering the workflow against persisted rows — completed tasks simply have output, so they don’t re-execute (Durability).
- Score outputs: scorers and eval suites receive parsed objects, not transcripts.
- Replay and fork from a checkpoint (Time Travel): a frame is meaningful because the state at that frame is typed rows, not conversation history.
Typed inputs: OpenAPI-derived tools
The same discipline runs in the other direction.createOpenApiTools compiles an OpenAPI 3.x spec into AI SDK tools — one per operation, Zod schemas converted from the spec’s JSON schemas — so the agent’s calls into your systems are typed too. Curate the surface with include/exclude/operations, and generate a reusable module from the CLI:
Guarantees and limits
- Guaranteed: a persisted output row always matches its Zod schema. There is no code path that writes an invalid row; consumers can trust
row.approvedexists and is boolean. - Not guaranteed: semantic correctness. Schema validation catches shape, not lies — pair it with scorers for quality.
- Outputs must be JSON objects. Plain text is rejected with a pointed error; wrap it as
z.object({ text: z.string() })and readdeps.<task>.text. - Repair costs real tokens. Each correction is another model call. A schema the model chronically misses (deeply nested, ambiguous field names) burns budget — flatten it.
- Constrained decoding only where the provider supports it; everywhere else it’s extraction + repair, which is robust but probabilistic. The 12-chances arithmetic above is a budget, not a proof.
- Enum/format edge cases in provider-native structured output vary by vendor; Smithers surfaces the provider error with a compatibility hint rather than papering over it.
Operating it
get_node_detail, get_run, list_artifacts.
See also
- How It Works — Tasks: three modes — the canonical task/output reference
- OpenAPI Tools — typed agent-to-API surface
- Durability — what typed rows buy you when the process dies
- Evals and Optimization — scoring beyond schema validity