Compile Pipeline

Compilation is the process that turns a WorkflowDefinition into a hashed, immutable WorkflowVersion that the runtime can execute and replay. In this milestone, compilation happens inside a planner harness session — a full agent loop with bash, memory, scratch, and skills — rather than via a single opaque draft() call.

Inputs And Outputs

// Dev-facing entry point (unchanged):
const result = await runWorkflow(myWorkflow, input, {
  world,
  tools,
});

Internally, runWorkflow calls the planner harness:

const planResult = await workflow.planner.harness.run({
  kind: "plan",
  workflowSnapshot,           // WorkflowDefinitionSnapshot (id, schemas, models, tools)
  input,
}, ctx);
// → planResult.lwir: the validated LWIR

The result is fully derived from inputs. The same definition + same input + a deterministic planner = the same workflowVersionId and workflowVersionHash. Re-running the same runId fast-forwards via event replay without calling the model again (see Memoization).

Two Identity Primitives

This milestone introduces a second identity hash alongside WorkflowVersion:

WorkflowDefinitionHash is stable across runs as long as the definition doesn't change. It is used in the PlannerManifest, OrchestratorManifest (which references each available workflow's hash), and drift detection (see Identity + Manifests).

WorkflowVersion varies per run because LWIR depends on input and planner non-determinism. Two compiles producing byte-identical LWIR collapse to the same WorkflowVersion (content-addressed).

Stage Overview

Stage 1: Normalize Schemas

Same as before: JSON Schema descriptors, Zod schemas, AI SDK output helpers all funnel through normalizeSchema() and normalizeOutputMode() in schema.ts into a closed NormalizedSchemaDescriptor.

Zod schemas are converted with z.toJSONSchema(). Live validator functions are rejected — LWIR must never lock executable validation code.

Stage 2: Validate Input

The supplied input is validated against the normalized input schema with Ajv (strict: false, allErrors: true). Failure throws WorkflowInputValidationError, which runWorkflow() converts into RunFailed(causeCode: "input_schema_error").

Stage 3: Snapshot Capabilities

The compiler builds a WorkflowDefinitionSnapshot — the view of the definition the planner harness receives:

type WorkflowDefinitionSnapshot = {
  readonly id: string;
  readonly description: string;
  readonly inputSchemaHash: string;
  readonly outputSchemaHash: string;
  readonly modelSlots: readonly { id: string; providerId: string; modelId: string; description: string }[];
  readonly plannerConfig: {
    readonly modelSlotId: string;
    readonly harnessId: string;
    readonly systemHash: string;
    readonly skillsHash: string;
  };
  readonly workerConfig: { readonly harnessId: string; readonly skillsHash: string };
  readonly globalTools: readonly string[];   // sorted tool ids
  readonly memoryConfig: { workflow: "rw" | "ro" | "none"; org: "rw" | "ro" | "none"; attached: readonly {...}[] };
};

Key invariants (updated vs prior alpha):

• Model slot IDs are derived from model names (e.g., gpt-4o-mini) with animal-suffix disambiguation on collision — no more qualityTier-based names like model.fast.
• The planner's own model is in plannerConfig.modelSlotId, separate from the execution modelSlots.
• Tool snapshots: every name in globalTools must exist in the registry; missing names throw WorkflowMissingToolError. Tool descriptions + inputSchema + outputSchema are hashed into the manifest (runtime compares at execution time, fails with capability_drift on mismatch).

Stage 4: Compute PlannerManifest

The PlannerManifest is computed before the harness session starts:

type PlannerManifest = {
  harnessId: string;                    // "workflowHarness@1.0.0"
  plannerModelSlotId: string;
  systemPromptHash: string;
  skillsHash: string;
  workflowDefinitionHash: string;
  globalToolsHash: string;
  memoryStoreIds: ReadonlyArray<string>;
  outerLoopContextHash?: string;        // present for re-plans after supervise
};

manifestHash = sha256(canonicalJson(PlannerManifest)). Stored in harness.session.started at session start.

Stage 5: Drift Check On Resume

If this runId has a prior harness.session.started event for the planner:

const stored = findEvent(events, "harness.session.started", { role: "planner" });
const currentHash = sha256(canonicalJson(currentPlannerManifest));
if (stored.payload.manifestHash !== currentHash) {
  throw new CapabilityDriftError({ stored, current, diff });
}

The planner session is fast-forwarded from events (see Memoization) rather than re-run.

Stage 6: Planner Harness Session

The planner harness session is a full agent loop. The default workflowHarness calls AI SDK streamText when available, otherwise generateText:

workflowHarness.run({
  kind: "plan",
  workflowSnapshot,
  input,
}, ctx)

The session has access to:

• bash — cwd-scoped to /mnt/scratch/own/. The planner can write draft LWIR to scratch, run node to test code bodies, and iterate.
• Memory — /mnt/memory/workflow/ (rw by default), /mnt/memory/org/ (ro by default). Planner reads prior lessons when designing.
• Skills — skill frontmatter injected into the system prompt; bodies readable via cat .agents/skills/<name>/SKILL.md.
• Tool descriptors (read-only for planning) — planner sees tool descriptions + schemas to write valid tool.call steps but does not execute tools.

The planner iterates within its own agent loop until it produces a valid LWIR. Validation happens inside the harness session (the model uses bash to test code bodies, introspects schemas, repairs its own output). The outer compile pipeline sees one final LWIR per planner session.

The planner lifecycle events from the prior alpha (PlannerStarted, PlannerDraftedWorkflow, WorkflowValidationFailed, WorkflowValidationSucceeded) are now represented as harness.model.called, harness.model.responded, harness.tool_call.started, and harness.tool_call.succeeded events inside the planner's session. OrchestrationRequested and WorkflowVersionRegistered remain as run-level events.

Stage 7: LWIR Validation

When the planner's agent loop commits a terminal output, validateLwir() from lwir.ts runs. The binding pass then ties the LWIR back to the snapshot:

• metadata.name must equal the workflow's id.
• sha256(input.schema) must equal the snapshot's input schema hash.
• sha256(output.schema) must equal the snapshot's output schema hash.
• permissions.models / permissions.tools must be subsets of the capability snapshot.
• Every ai.* step with.model must be in the allowed model slot set.
• Every tool.call step with.tool must be in the allowed tools.

Findings have a stable shape:

{
  "severity": "error",
  "code": "binding.input_schema_mismatch",
  "path": "$.input.schema",
  "message": "LWIR input schema must match the workflow definition's input schema."
}

If validation fails, the harness's agent loop receives the findings and repairs (the model iterates again within the same session). The outer compile pipeline sees the final valid LWIR only.

Stage 8: Register And Lock

When validation passes, registerWorkflowVersion(lwir) produces a WorkflowVersion:

{
  "id": "wfver_4f0aab7d12345678",
  "hash": "sha256:...",
  "canonicalizer": "little-workflow-canonical-json@alpha",
  "canonicalJson": "{\"apiVersion\":\"littleworkflow.dev/v0.1\",...}",
  "lwir": { ... }
}

WorkflowVersion = sha256(canonicalJson(LWIR)) — unchanged.

compiler-lock.ts wraps it with a lock:

{
  "workflowVersionId": "wfv_4f0aab7d12345678",
  "workflowVersionHash": "sha256:...",
  "lwirVersionId": "wfver_4f0aab7d12345678",
  "lwirHash": "sha256:...",
  "workflowDefinitionHash": "sha256:...",
  "inputSchemaHash": "sha256:...",
  "requestedOutput": { "mode": "object", "schema": { ... } },
  "requestedOutputHash": "sha256:...",
  "capabilityManifest": { ... },
  "capabilityManifestHash": "sha256:...",
  "modelSlots": [ ... ],
  "tools": [],
  "validationHash": "sha256:..."
}

Memoization

The planner harness session is durable — every model call and tool call within the session is event-logged. On resume with the same runId:

1. Events are replayed — completed harness.model.called, harness.model.responded pairs fast-forward without re-calling the model.
2. Completed harness.tool_call.started, harness.tool_call.succeeded pairs replay their results.
3. Only the in-flight call (the one with no terminal event) is re-executed.

This is identical to LWIR per-step memoization, applied at the harness session level. Re-running the same (runId, workflowDefinition, input) is deterministic via event replay.

Outer-Loop Context

When maxOuterCycles > 1, the planner harness session's PlannerManifest carries an outerLoopContextHash (sha256 of prior cycles). Each cycle's manifest differs → each cycle's manifestHash differs → resume correctly distinguishes which cycle's session to fast-forward.

The planner's agent loop receives prior cycle information in the task context:

type PlanTask = {
  readonly kind: "plan";
  readonly workflowSnapshot: WorkflowDefinitionSnapshot;
  readonly input: unknown;
  readonly outerLoopContext?: {
    readonly cycleNumber: number;
    readonly maxCycles: number;
    readonly isFinalCycle: boolean;
    readonly priorCycles: ReadonlyArray<{ cycleNumber; status; output; summary? }>;
  };
};

What This Buys

• The planner can test code before emitting LWIR — iterate in bash scratch, read prior lessons from memory, embed validated code bodies.
• The same (runId, definition, input) always yields the same LWIR (fast-forwarded from event cache after first compile).
• The runtime verifies the LWIR matches the lock before running a step — no trust in the planner is required at runtime.
• WorkflowDefinitionHash (new) provides stable identity across many LWIR compiles of the same definition, enabling orchestrator-level drift detection.

Worked Example: support.summarize-ticket

A concrete trace of one workflow through the compile pipeline. Every JSON snippet here is the shape the alpha SDK produces; the contents are abbreviated for readability.

import { defineWorkflow, model } from "@little-stack/workflow";
import { openai } from "@ai-sdk/openai";
import { z } from "zod";

const summarizeTicket = defineWorkflow({
  id: "support.summarize-ticket",
  description: "Summarize a support ticket and classify its severity.",

  input: z.object({
    ticketId: z.string(),
    transcript: z.string(),
  }),

  output: z.object({
    summary: z.string(),
    severity: z.enum(["low", "medium", "high"]),
  }),

  // Execution model slots (the planner chooses among these in LWIR step with.model)
  models: [
    model(openai("gpt-5"), {
      description: "Strong reasoning model for complex analysis.",
    }),
    model(openai("gpt-4o-mini"), {
      description: "Fast model for short structured outputs.",
    }),
    // Slot ids auto-derived: "gpt-5" (first wins clean name), "gpt-4o-mini"
  ],

  // Planner — required
  planner: {
    model: openai("gpt-5"),      // planner's own model (separate from execution models)
    system: "You design support-ticket workflows. Test code in scratch before committing.",
  },
});

The planner harness session receives the WorkflowDefinitionSnapshot (Stage 3 above) plus the run input as its task, then iterates in its agent loop. A typical first LWIR (abbreviated):

{
  "apiVersion": "littleworkflow.dev/v0.1",
  "kind": "Workflow",
  "metadata": {
    "name": "support.summarize-ticket",
    "version": "0.1.0",
    "description": "Summarize a support ticket and classify severity."
  },
  "input": {
    "schema": {
      "type": "object",
      "required": ["ticketId", "transcript"],
      "properties": {
        "ticketId": { "type": "string" },
        "transcript": { "type": "string" }
      }
    }
  },
  "output": {
    "schema": {
      "type": "object",
      "required": ["summary", "severity"],
      "properties": {
        "summary": { "type": "string" },
        "severity": { "type": "string", "enum": ["low", "medium", "high"] }
      }
    }
  },
  "permissions": {
    "models": ["gpt-4o-mini"],
    "tools": [],
    "secrets": [],
    "network": []
  },
  "steps": [
    {
      "id": "summarize",
      "uses": "ai.generate",
      "needs": [],
      "with": {
        "model": "gpt-4o-mini",
        "prompt": "Summarize ticket {{ input.ticketId }} based on this transcript:\n{{ input.transcript }}\nReturn a 1-2 sentence summary and a severity from low|medium|high."
      },
      "output": {
        "mode": "object",
        "schema": {
          "type": "object",
          "required": ["summary", "severity"],
          "properties": {
            "summary": { "type": "string" },
            "severity": { "type": "string", "enum": ["low", "medium", "high"] }
          }
        }
      }
    }
  ]
}

Notes about the LWIR shape:

• permissions.models uses slot ids ("gpt-4o-mini") — the derivation rule, not qualityTier names.
• step.uses is ai.generate.
• onFailure.fixer is an optional field on code.run steps (see below).

onFailure.fixer Field

code.run steps can optionally declare a fixer for self-recovery on failure:

{
  "id": "compute-score",
  "uses": "code.run",
  "with": {
    "entrypoint": "score.ts",
    "files": {
      "score.ts": {
        "content": "export async function run({ input }) {\n  return { score: input.signals.length / 10 };\n}",
        "sha256": "sha256:1ed396c7cbeba5c323250026707a76c1e1ad3f62572334094e5167c357c6de7c"
      }
    },
    "sandbox": { "network": "deny", "env": "deny", "fs": "deny" }
  },
  "output": {
    "mode": "object",
    "schema": { "type": "object", "properties": { "score": { "type": "number" } } }
  },
  "onFailure": {
    "fixer": {
      "model": "gpt-4o-mini",      // slot id from workflow.models
      "maxAttempts": 3,
      "system": "Fix the code so it returns output matching the schema. Test in scratch."
    }
  }
}

When a code.run step fails, the runtime checks for onFailure.fixer. If declared, a fixer harness session (kind: "fix_step") is spawned. The fixer has the failed code body, the error, and the output schema. On success, harness.session.completed.fixedSource carries the repaired code. See Fixer.

What This Means For The Planner

• The planner is never in the loop at runtime. Its job ends once registration succeeds; worker harness sessions execute the LWIR.
• The planner's output is hashable JSON. The runtime never parses natural language to dispatch a step.
• The planner CAN use bash, memory, and skills during design — a capability the prior function-only draft() call did not provide.
• The planner is allowed to iterate internally. Findings are structured; the repair loop is within the planner's own agent loop.

Inspecting Compile Stages

After a runWorkflow call, the event log contains:

harness.session.started   (role: "planner")
harness.model.called      (turn 1 — planner reasons about the workflow)
harness.model.responded   (turn 1)
harness.tool_call.started  (caller: "model", bash — tests code body in scratch)
harness.tool_call.succeeded
harness.model.called      (turn 2 — refines LWIR)
harness.model.responded   (turn 2 — produces valid LWIR)
harness.session.completed
WorkflowVersionRegistered
RunStarted

little events <run-id> surfaces the full sequence. See Event Log for payload shapes.