Monitor workflows

A workflow that runs long, unattended, or in a loop can wedge silently and nobody notices until morning. A monitor is a Smithers workflow whose job is to watch one other run: it samples the run’s health on an interval, classifies what it sees, and either applies a small reversible repair or escalates to a human. Monitors are opt-in per workflow and cost nothing until you write one. A workspace with no monitor files behaves exactly as it did before.

10-second quickstart

Write the monitor at .smithers/monitor/<workflowId>.tsx, named after the workflow it watches:

.smithers/
├── workflows/
│   └── nightly.tsx        # the workflow
└── monitor/
    └── nightly.tsx        # the monitor that watches every `nightly` run

bunx smithers-orchestrator up .smithers/workflows/nightly.tsx   # auto-launches .smithers/monitor/nightly.tsx
bunx smithers-orchestrator up nightly.tsx --monitor ops/watch.tsx  # pick a monitor explicitly
bunx smithers-orchestrator up nightly.tsx --no-monitor             # opt out

The flag lives on every launch path that starts a run: up, workflow run, and the overridable oneshot workflow.

What a monitor is

A monitor is an ordinary workflow. Nothing about it is special except where the file lives and how it is launched:

It starts with the watched run, as a sibling run executing in parallel.
Its parent_run_id is the watched run, so bunx smithers-orchestrator ps and bunx smithers-orchestrator inspect RUN_ID show the pairing, the Gateway can render it, and bunx smithers-orchestrator cancel RUN_ID cascades to the monitor.
It is torn down when the watched run finishes, so a monitor never outlives what it watches.
Its run id is <watchedRunId>-monitor, and it carries a smithersMonitorFor annotation naming the watched run.
A monitor never gets a monitor of its own. Three independent guards enforce that: the file’s location under monitor/, the --no-monitor flag it is launched with, and the SMITHERS_MONITOR_SUPPRESS environment variable set on its process.

Smithers passes the monitor its subject as run input:

{ watchRunId: string, watchWorkflowId: string, watchWorkflowPath: string }

Writing one

Compose the shipped <Monitor> component rather than hand-rolling a poll loop. A monitor file is a few lines:

/** @jsxImportSource smithers-orchestrator */
import { Monitor, createSmithers } from "smithers-orchestrator";
import { z } from "zod/v4";
import { agents } from "../agents";

const { Workflow, smithers, outputs } = createSmithers({
  health: z.object({
    condition: z.enum([
      "healthy",
      "stalled",
      "wedged-node",
      "runaway-loop",
      "awaiting-human",
      "failing",
      "unknown",
    ]),
    runStatus: z.string(),
    targetNodeId: z.string().nullable().default(null),
    evidence: z.string(),
    summary: z.string(),
  }),
  action: z.object({ action: z.string(), changed: z.boolean().default(false), summary: z.string() }),
});

export default smithers((ctx) => (
  <Workflow name="nightly-monitor">
    <Monitor
      watchRunId={ctx.input.watchRunId}
      agent={agents.cheapFast}
      healthOutput={outputs.health}
      actionOutput={outputs.action}
      intervalMs={60_000}
    />
  </Workflow>
));

nodeId, runId, and iteration are reserved output columns, which is why the implicated node is reported as targetNodeId.

How the heartbeat works

Each beat is three steps, and the first beat samples immediately rather than sleeping first:

A durable <Timer> paces the loop.
One agent task samples the watched run and classifies it into exactly one condition.
A <DecisionTable> routes that condition to its handler, first match wins.

The loop exits when the watched run reaches a terminal status, or after maxChecks beats.

The condition set

The classifier must pick exactly one, which is what makes routing deterministic:

Condition	What it means	Default handling
`healthy`	Progressing, or finished cleanly	Nothing. Keep watching.
`stalled`	Silent past the stall budget, nothing waiting on a human	Auto-heal: resume the run
`wedged-node`	One node burning attempts on the same error	Auto-heal: retry that node once
`runaway-loop`	Iterations climbing without converging	Escalate to a human
`awaiting-human`	Parked on an approval or human request	Report it. Never answer for the human.
`failing`	The run failed terminally	Report the failure and evidence
`unknown`	Evidence unreadable or contradictory	Escalate to a human

What a monitor may do on its own

The shipped defaults are deliberately timid, because a monitor that guesses wrong makes an incident worse. Only stalled and wedged-node heal without a human, and only because both repairs are idempotent and reversible: resuming re-enters the same durable frame, and retrying a node adds an attempt without discarding prior state. Running either twice is harmless. Everything else escalates through a durable human request, which shows up in bunx smithers-orchestrator human list and in the Gateway. Cancelling, rewinding, reverting, time-travelling, resolving an approval on a human’s behalf, and editing the repository are never done autonomously. Widen that authority explicitly when you mean to:

<Monitor autoHeal={["stalled", "wedged-node", "runaway-loop"]} ... />

Adding runaway-loop swaps its handler from “escalate” to “cancel the run”, which is destructive by design and therefore never a default. Replace any handler outright, or silence one with null:

<Monitor
  handlers={{
    failing: <Task id="page-oncall" output={outputs.action} agent={agents.cheapFast}>Page the on-call engineer.</Task>,
    "awaiting-human": null,
  }}
  ...
/>

Reading run state

A monitor reads the watched run through the Gateway client (smithers-orchestrator/gateway-client) or the public CLI (status, inspect, events, node, why, ps). It must never open the store directly: .smithers/*.db, .smithers/pg, and smithers.db are private engine state, and reading them races the very run being watched. The prompt shipped with <Monitor> binds the agent to that rule, along with what healthy and unhealthy look like, what evidence to gather before acting, and when to escalate instead of guessing. Extend it with repo-specific doctrine rather than replacing it:

<Monitor guidance="Runs here routinely sit on one agent task for 20+ minutes. Streaming events mean healthy, however long it has been." ... />

Pass prompt only when you need to replace the doctrine entirely.

Testing a monitor

A monitor is a workflow, so it tests like one, with coverWorkflow or renderWorkflow from smithers-orchestrator/testing. Mock the classifier’s verdict and assert which handler ran:

const result = await coverWorkflow(monitorModule, {
  input: { watchRunId: "run-under-watch" },
  maxLoopIterations: 1,
  mocks: {
    "monitor-check": { condition: "stalled", runStatus: "running", targetNodeId: "implement", evidence: "...", summary: "..." },
    "monitor-stalled": { action: "resume", changed: true, summary: "resumed" },
  },
});
expect(result.executed).toEqual(["monitor-check", "monitor-stalled"]);

See examples/monitor-workflow.jsx for a complete, runnable monitor.

Start

Articles

Learn

Build Workflows

Run and Operate

Workflow Pack

Components

Integrations

Agent Support

Examples

Contributing

Monitor workflows

10-second quickstart

What a monitor is

Writing one

How the heartbeat works

The condition set

What a monitor may do on its own

Reading run state

Testing a monitor

​10-second quickstart

​What a monitor is

​Writing one

​How the heartbeat works

​The condition set

​What a monitor may do on its own

​Reading run state

​Testing a monitor

10-second quickstart

What a monitor is

Writing one

How the heartbeat works

The condition set

What a monitor may do on its own

Reading run state

Testing a monitor