> ## Documentation Index
> Fetch the complete documentation index at: https://smithers.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Testing workflows

> Simulate workflows, script agents, and assert task behavior with Bun tests.

`@smithers-orchestrator/testing` is the consumer-facing test surface for Smithers workflows. Import it from the published facade:

```ts theme={"theme":{"light":"github-light","dark":"github-dark"}}
import {
  auto,
  dryRun,
  fakeAgent,
  renderPrompt,
  renderWorkflow,
  runTask,
  simMatchers,
  simulate,
  toHaveExecuted,
  toHaveExecutedInOrder,
  toHaveFinished,
} from "smithers-orchestrator/testing";
```

<Note>
  The examples below use the normal `createSmithers` workflow API and `bun:test`; the testing helpers do not replace workflow authoring components.
</Note>

## Unit-test a workflow

`simulate(workflow, options?)` renders and runs a workflow in memory. Pass the workflow's input through `input`, and provide task responses in `mocks`. Mock keys can be exact task IDs or globs; `"*"` is the fallback key for agent tasks. Each value can be a function, a raw output value, `auto`, or a `FakeAgent`. Agent tasks without a matching mock fail loudly, which keeps a test from accidentally calling a real agent.

The returned `Sim` is lazy: call `await sim.run()` to execute it. It records the task IDs it ran, the values stored in each output table, and the final run status.

```tsx theme={"theme":{"light":"github-light","dark":"github-dark"}}
/** @jsxImportSource smithers-orchestrator */
import { describe, expect, test } from "bun:test";
import { z } from "zod";
import { createSmithers } from "smithers-orchestrator";
import { auto, fakeAgent, simulate } from "smithers-orchestrator/testing";

const schemas = {
  input: z.object({ name: z.string() }),
  greeting: z.object({ message: z.string() }),
  reply: z.object({ text: z.string() }),
};

function buildWorkflow() {
  const { Workflow, Task, smithers, outputs } = createSmithers(schemas);
  const workflowAgent = fakeAgent(schemas.greeting, {
    output: { message: "workflow fallback" },
  });

  return smithers((ctx) => (
    <Workflow name="greeting">
      <Task
        id="greet"
        output={outputs.greeting}
        agent={workflowAgent}
        noRetry
      >
        {`Greet ${ctx.input.name}`}
      </Task>
      <Task id="reply" output={outputs.reply} dependsOn={["greet"]}>
        {() => ({ text: `done ${ctx.input.name}` })}
      </Task>
    </Workflow>
  ));
}

describe("greeting workflow", () => {
  test("runs with a scripted agent", async () => {
    const scripted = fakeAgent.sequence(
      schemas.greeting,
      [{ message: "hello Ada" }],
    );

    const sim = simulate(buildWorkflow(), {
      input: { name: "Ada" },
      mocks: { greet: scripted },
    });

    await sim.run();

    expect(sim.status).toBe("finished");
    expect(sim.executed).toEqual(["greet", "reply"]);
    expect(sim.outputs.greeting).toEqual([{ message: "hello Ada" }]);
    expect(sim.outputs.reply).toEqual([{ text: "done Ada" }]);
    expect(scripted.calls).toHaveLength(1);
  });

  test("can generate a schema-shaped response automatically", async () => {
    const agent = fakeAgent(schemas.greeting, auto);

    await expect(agent.generate()).resolves.toEqual({
      output: { message: "string" },
    });
  });
});
```

The main `Sim` fields are:

| Field      | Meaning                                                                                     |
| ---------- | ------------------------------------------------------------------------------------------- |
| `executed` | Task node IDs in the order they executed.                                                   |
| `outputs`  | Arrays of validated task outputs keyed by output-table name, such as `greeting` or `reply`. |
| `status`   | The current run status, including `pending`, `running`, `finished`, or `failed`.            |

`sim.output` is the latest workflow output. `sim.task(id)` returns that task's status, outputs, and rendered prompts; `unusedMocks` identifies mock keys that were never consumed. A failed simulation exposes `error` and `sim.run()` rejects. Output schemas are checked before values are recorded.

### Scripted agents

`fakeAgent(schema, script, options?)` creates a `FakeAgent` whose script can be a bare schema value, a `{ output, text, files }` response, `auto`, or a function returning one of those forms. Responses with an `output` are validated against the supplied schema. `fakeAgent.sequence(schema, entries, options?)` consumes a fixed list in order, which is useful for retries or repeated task runs.

The `FakeAgent<T>` type describes the returned agent. Its `generate()` method is compatible with Smithers agents, while `calls`, `lastPrompt()`, and `reset()` let a test inspect or clear its history. The optional agent settings are `id`, `model`, and `supportsNativeStructuredOutput`.

`auto` is a schema-derived response sentinel. It requires an output schema and supplies an example-shaped output; use an explicit value or function when the exact result matters. File responses are written below the `rootDir` supplied to `generate()` or the test helper.

## Render and run one task

Use `renderWorkflow` when a test needs a rendered frame or task descriptors without executing them. It resolves task output schemas and attaches the same subflow and sandbox compute functions as the engine. The returned `RenderedWorkflow` includes the rendered `tasks`, `ctx`, `runId`, `frameNo`, and `toXml()`.

```ts theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { renderPrompt, renderWorkflow, runTask } from "smithers-orchestrator/testing";

const frame = await renderWorkflow(buildWorkflow(), {
  input: { name: "Ada" },
  runId: "render-test",
});
const task = frame.tasks.find(({ nodeId }) => nodeId === "greet");
if (!task) throw new Error("greet task was not rendered");

const prompt = renderPrompt(task.prompt);
const output = await runTask(task, { runId: frame.runId });

console.log({ prompt, output, xml: frame.toXml() });
```

`renderPrompt(prompt)` converts a rendered task prompt to the same text form used by the production task renderer. `runTask(task, options?)` executes only one `TaskDescriptor`: static descriptors return their payload, compute descriptors call their compute function, and agent descriptors call their selected agent. When the descriptor has an output schema, the result is validated. For agent descriptors, `RunTaskOptions` forwards `rootDir`, `attempt`, and `runId` to the agent call.

`dryRun(ast, options?)` is the planning companion for the durability scenario API below. It canonicalizes and compiles the scenario, reports required capabilities and admissions, creates a replay-bundle skeleton, lists planned steps, and sets `executesAgents` to `false`. Its returned `run()` function is the point at which the scenario is actually executed.

## Assertions with Bun matchers

The testing package augments `bun:test`'s matcher types. Register the exported matcher set once in the test file, then assert against a `Sim`:

```ts theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { expect, test } from "bun:test";
import { simMatchers, simulate } from "smithers-orchestrator/testing";

expect.extend(simMatchers);

test("checks the simulated path", async () => {
  const sim = simulate(buildWorkflow(), {
    input: { name: "Ada" },
    mocks: { greet: { message: "hello Ada" } },
  });

  await sim.run();

  expect(sim).toHaveExecuted(["greet", "reply"]);
  expect(sim).toHaveExecutedInOrder(["greet", "reply"]);
  expect(sim).toHaveFinished();
});
```

`toHaveExecuted(ids)` requires every listed ID to appear. `toHaveExecutedInOrder(ids)` requires them as an ordered subsequence, so unrelated tasks may occur between them. `toHaveFinished()` checks for the exact `"finished"` status. The same functions are also exported as `toHaveExecuted`, `toHaveExecutedInOrder`, and `toHaveFinished` for direct matcher use.

## Advanced: durability scenario harness

For modeled scheduling and durability behavior, compose an immutable scenario AST with `scenario`, `step`, and `fault`; `barrier` and `extension` are available for explicit synchronization and registered extensions. A `step` can declare `input`, `dependsOn`, `capabilities`, and a `run` function. Its task runtime exposes mediated `effect`, virtual `sleep`, structured `log`, and explicitly opaque work. A `fault` names a phase and operation at which to inject a failure or ambiguity.

`runScenario(ast, options?)` executes the AST and returns a result containing its status, outputs, trace, replay identity, control log, capability report, ambiguity records, and determinism report. `unitSimHarness`, `integrationHarness`, and `e2eHarness` select the capability tier: deterministic virtual-time and seeded-interleaving simulation; a real database adapter; or a real process adapter. `runScenario` defaults to `unitSimHarness()`; the real tiers require verified, executable production adapters and can report a capability failure or skip according to policy.

<Warning>
  Unit simulation is not a database or child-process substitute. External effects remain at-least-once unless the application supplies idempotency, and effects outside the task-runtime mediation boundary are opaque. See [`packages/testing/README.md`](https://github.com/smithersai/smithers/blob/main/packages/testing/README.md) for the real-system boundary, replay, fault, and adapter details.
</Warning>

## Related evaluation tools

For LLM-judge assertions, see [`llmJudge`](/reference/scorers#llmjudge) and [`runScorersBatch`](/reference/scorers#runscorersbatch) in the [Scorers reference](/reference/scorers). For dataset-driven regression suites, see the [Eval Suites Quickstart](/guides/evals-quickstart).