Little documentation
Guides and APIInternals

Local World Storage

How the alpha world stores events, artifact manifests, blobs, and run locks on the filesystem.

Little Workflow: published on npm as little-workflow@0.2.0-alpha.1 (2026-10-06). Sections labeled UNRELEASED describe source-only changes. Release status

localWorld({ dataDir }) is the disk adapter: SQLite events, content-addressed artifacts and run-execution locks. virtualWorld() uses a supplied filesystem. Both share the event rules in world.ts; storage adapters provide SQLite rows or published event files, and RunFiles handles the remaining run-owned I/O. The layout below describes the disk adapter.

Directory Layout

<dataDir>/
  events/
    events.db                  # SQLite event store
    events.db-wal              # WAL sidecar, when active
    events.db-shm              # SQLite shared-memory sidecar, when active
  artifacts/
    <artifactId>.json          # manifest
    blobs/
      <artifactId>.bin         # payload bytes
  locks/
    <sha256(runId)>/owner.json
    <sha256({ outerLoopId })>/owner.json   # outer-loop fence
  outer-loops/
    <outerLoopId>.json         # outer-loop session manifest
  • events/ — SQLite/WAL event store keyed by run id and sequence. See Event Log.
  • artifacts/ — one manifest per artifact, paired with the raw blob in artifacts/blobs/. Content-addressed by sha256 of the payload.
  • locks/ — directory-based locks for run execution fencing, plus one per outer loop that serializes callers sharing an outerLoopId. Each lock holds an owner.json with { pid, token, createdAt }; a lock whose owner process is gone is recovered.
  • outer-loops/ — one JSON manifest per outer-loop session. Written and updated transactionally by the outer-loop scheduler.

The data directory defaults to .little-workflow in the current working directory.

Outer-Loop Manifest

outer-loops/{outerLoopId}.json is the cycle-level durability record. It is written via atomic rename (write to *.tmp, fsync, rename into place) after every OuterLoopCycleCompleted event.

{
  "outerLoopId": "ol_…",
  "goal": { "workflowDefinitionHash": "…", "description": "…" },
  "maxCycles": 5,
  "inputHash": "sha256:…",   // identity of the loop's input; a resume with other input is rejected
  "cycles": [
    {
      "cycleNumber": 1,
      "runId": "run_…",
      "workflowVersionId": "wv_…",
      "status": "completed",
      "output": { }
    }
  ],
  "pendingCycleRunId": "run_…",   // set before a cycle starts; cleared after supervise
  "result": { "kind": "done", "finalOutput": { } }  // set when supervise returns done
}

pendingCycleRunId is the key field for crash recovery. Its presence on resume means a cycle was in flight; the runtime reuses that runId so that Layer A replay can resume the cycle from its last committed step event.

Run Identifiers

  • runId — run_<1..80 chars of [A-Za-z0-9_-]>. The CLI passes --run-id; the SDK generates a UUID-based default. WorldPathError is thrown on malformed IDs to keep arbitrary paths out of events/.
  • artifactId — art_<16 hex> derived from sha256({ runId, stepPath, name, contentType, encoding, payloadHash }). Same inputs → same artifactId → idempotent writes.
  • artifactRef — artifact://<artifactId>. The string form used in events.

Write Serialization

world.ts uses an in-process write chain to keep concurrent writes deterministic:

const writeChains = new Map<string, Promise<unknown>>();

function withWriteChain(key, fn) {
  const previous = writeChains.get(key) ?? Promise.resolve();
  const next = previous.catch(() => undefined).then(fn);
  writeChains.set(key, next);
  return next.finally(/* cleanup */);
}
  • For events the key is events:${resolve(dataDir)}:${runId}.
  • For artifacts the key is artifact:<dataDir>:<artifactId>.

Two callers in the same process target the same run id ⇒ they queue. The runtime's in-process fence keeps two runs from racing the same run id, and the filesystem lock keeps two processes from racing the same data directory.

Harness event rows also carry the emitter's appendIdempotencyKey and a canonical request digest. SQLite enforces one non-null key per run. An identical retry returns the already committed row; the same key with different content is rejected. This closes the lost-response window without changing the keyless append contract for generic Workflow facts.

Artifact Pipeline

100%

Three pieces matter:

  • Content addressing. Writing the same payload twice does not produce two artifacts; the artifactId is a deterministic hash of identity + content. The second write detects the existing manifest, verifies the blob, and returns.
  • Atomic publish. Both blob and manifest are written to *.tmp files first and renamed into place. A crash mid-write leaves a .tmp file the next process can ignore.
  • Integrity verification. Every readArtifact() re-hashes the bytes and compares with the manifest's sha256. Mismatch raises ArtifactHashMismatchError. Mismatched sizeBytes raises ArtifactManifestCorruptError. These are non-recoverable; the runtime treats them as RuntimeIntegrityError.

Manifest Shape

{
  "artifactId": "art_4f0a2c81e7b09a3d",
  "artifactRef": "artifact://art_4f0a2c81e7b09a3d",
  "runId": "run_killresume_abcdef",
  "stepPath": "summarize",
  "name": "output",
  "contentType": "application/json",
  "encoding": "json",
  "sha256": "sha256:...",
  "sizeBytes": 412,
  "createdAt": "2026-05-19T18:43:11.452Z"
}

encoding is one of three deliberately narrow values:

EncodingPayload typeBytes
jsonobject / array / number / boolean / nullcanonicalJson(value)
utf-8stringUTF-8 bytes
binaryUint8Arrayraw bytes

That keeps round-tripping deterministic and avoids the "is this JSON or just a string that happens to look like JSON" footgun.

Lock Pipeline

Run locks live in <dataDir>/locks/<lockName>/ where lockName = sha256(runId). The directory's existence is the lock; an owner.json inside carries { pid, token, createdAt }.

100%

Notes:

  • The recovery path is its own keyed lock — recovery cannot stomp on a concurrent recovery.
  • Stale detection uses process.kill(pid, 0) plus a grace window (LOCK_OWNER_GRACE_MS = 30s) so the very first moments of an unflushed owner file don't trigger a takeover.
  • realpath() of dataDir is folded into the in-process key so two callers pointing at the same directory via different paths still serialize.

Materialized State

materializeRunState(world, runId) is the read path the CLI's little replay uses. It reads committed events with listEvents() and folds them through materializeRunStateFromEvents() from run-state.ts. The function is in world.ts so callers do not have to import run-state.ts directly.

Failure Modes And Errors

ErrorWhen
WorldPathErrorA run ID or artifact ID didn't match the allowed pattern.
RunNotFoundErrorlistEvents / materializeRunState saw an empty event file.
ArtifactNotFoundErrorNo manifest or blob for the ref.
ArtifactHashMismatchErrorBlob bytes don't match the manifest digest.
ArtifactManifestCorruptErrorManifest JSON is malformed or violates the manifest schema.
EventStoreCorruptErrorAn event line is not parseable.

All of these are unrecoverable at the world layer. The runtime treats them as RuntimeIntegrityErrors — they surface tampering or disk damage, not normal flow.

What Local World Is Not

  • Not multi-host. Two machines writing to the same dataDir is undefined; share filesystems are not a supported topology.
  • Not multi-tenant. Directory separation is your only isolation. Don't share a dataDir across customer runs.
  • Not a queue. The runtime fences each run; there is no broker for new runs.
  • Not a stream. Event tails are read by re-opening the file. WebSocket / Server-Sent-Events surfaces arrive with the inspector after alpha.

Production durability — Postgres event log, Redis queues, S3 / R2 blob storage — arrives via separate world adapters in v0.1-beta and later. The interface boundary is runtime.ts ↔ world.ts; everything you read here is meant to be one implementation behind that interface.

On this page