Execution Journal

A plan-scoped, project-keyed execution journal that acts as the durable shared-state layer for agent orchestration. The vault plan is the immutable spec (intent); the journal is the mutable record (execution). Together they close the intent-artifact traceability gap that is the core challenge of agent-assisted development: agent-written code outpaces mental-model formation, and the gap is in drift detection, coverage, and decision archaeology — not code quality.

This is the well-established blackboard / shared-state pattern for multi-agent orchestration, built purpose-fit for the herbylab stack rather than adopted from a heavyweight framework (LangGraph/CrewAI/AutoGen). Lightweight, Postgres-backed alongside Trellis and OB1, accessed via a small MCP surface.

Goal

Give Lovebug and its workers a durable, queryable execution record keyed to the immutable vault plan, so that:

Agents pass conclusions, not conversation context across session boundaries. A worker dies; the journal persists; the next agent reads a small focused slice instead of the dead worker's sprawling transcript.
Ruled-out approaches are captured explicitly, killing the re-proposal loop where a fresh agent re-suggests something a prior agent already tried and rejected.
Lovebug can answer "what is the state of project X" in a bounded number of tokens without scanning raw history or holding project state in its own context.
The full execution sequence remains auditable months later for decision archaeology, without re-reading worker transcripts.

Locked Decisions

Two stores, two shapes. Plans are documents (immutable, vault). Execution is records (mutable, Postgres). Different state wants different stores — same logic that produced Trellis (threads) and OB1 (semantic memory).
Plan is immutable; journal references it. The journal row links to the plan via foreign key to the projects table. Plan stays the frozen spec; journal is the running receipt.
Postgres, reusing existing infra. No new datastore. Schema alongside Trellis/OB1. Journal entries are small text — storage is a non-issue; retention is effectively forever, which is the point (decision archaeology).
MCP is the access surface, not the control surface. MCP is passive: it exposes read/write tools that agents call. It cannot spawn, reap, or swap workers. Control (lifecycle) lives in whatever has process/loop control — the Claude Code agent loop or an external harness. The journal MCP is the nervous system for information, not the hands.
Reads are shaped at the query layer, not filtered in-context. Filtering in-context burns context; filtering at the query layer does not. The MCP exposes narrow, purpose-shaped reads so agents pull exactly the slice they need.
Single writer per entry, write enforced deterministically. The "worker must write its handoff before it is considered done" rule belongs in deterministic code (e.g. a SubagentStop hook or harness wait/exit logic), not model goodwill — model-interpreted instructions degrade as context fills.

Open Items

Audit log vs. orchestration handoff are different artifacts. Two audiences want different things: the audit audience wants the full sequence; the orchestration audience wants what is true now and what the next actor must know. Leaning toward: structured handoff field (tight contract) + free-form log (whatever the worker writes). Normal-flow reads hit handoffs; logs are for drill-down. Bake the handoff field in from day one even if it starts as free text — retrofitting structure later is harder than enriching it.
Who writes the handoff — worker writes its own (fast, but biased to "what I did" not "what next needs") vs. orchestrator writes it (better framed, but reads raw worker output and bloats). Start by trusting the worker; add a synthesis pass only if handoffs prove too noisy. (Synthesis-layer design tracked separately.)
Task addressing scheme — heading path (readable, fragile), explicit task IDs (stable, needs discipline), or line/hash anchors (stable, opaque). Task IDs (e.g. T-042) preferred: they also feed worker briefs ("complete T-042") so brief → journal traceability falls out for free. Nudges plan structure toward spec-shaped, addressable tasks — aligned with the spec-driven workflow, not additional.
Workers write direct vs. via orchestrator — direct writes need worker creds + network to the MCP (easy if workers run on herbys-dev) but give per-worker fidelity; via-orchestrator is single-writer/simpler-auth but loses worker-internal decision detail.
Serial vs. parallel dispatch — "tasks in order" is fine for v1 (serial, simplest). Some plans have parallelizable independent chunks; defer fan-out until serial is proven.
Projection for rollup — if the journal grows and Lovebug wants "current state" without scanning every row, add a derived view (latest handoff per plan task + status). Postgres view or materialized view. Not needed day one.

Phases

Phase 1 — Schema + write path

Define journal table. Working field set: id, project_slug (FK to projects), plan_ref (task ID into the plan), actor (lovebug / worker-<id> / travis), action (verb-led, short), result (success / failed / partial / skipped / gated), log (free-form), handoff (structured: goal, current state, next step, ruled-out + why), artifacts (commits/files/logs/vault refs), started_at / ended_at, optional parent_id (nesting worker steps under an orchestrated parent).
Decide parent_id now or later — flat log vs. tree mirroring real decomposition. Tree helps debugging, adds write complexity.
Define the write contract: required fields, allowed result values, what handoff must contain, size discipline (handoff is conclusions, not the 8k tokens of tool output the worker generated to reach them). The contract matters more than the schema — a blackboard fails when agents write inconsistent mush.

Phase 2 — MCP read surface

log_step — write an entry.
get_task_context(slug, task_id) — narrow slice for one task: latest handoff, dependency handoffs, adjacent-task handoffs that touched related files/services. ~3–10 entries, the input to a worker brief.
get_project_state(slug) — bounded rollup for orchestration: which plan tasks done / in-flight / failed / blocked. A few hundred tokens. What Lovebug calls before deciding the next move.
(Optional) get_project_briefing(slug) — synthesized bounded state doc for hydrating a new agent onto a long-running project. Source of truth is the journal; briefing is the rendered view.

Phase 3 — Lifecycle integration

Wire the write into worker completion deterministically. If Claude Code subagents: SubagentStop hook writes the entry unconditionally. If external harness + CLI workers: write-on-exit before the subprocess returns.
Define the orchestrator loop against the journal: get_project_state → pick next task → build brief via get_task_context → spawn worker → worker writes entry + dies → loop. Orchestrator never reads raw worker output; context per cycle stays roughly constant regardless of project size.
Define the not-clean-return policy: worker returns failure/partial/gated → retry same brief / retry modified / escalate to Travis / skip. This is the one place the orchestrator needs real logic; define the policy upfront rather than improvising.

Notes

Why this matters for context. Models cannot dump-and-reload their own context — context is the prompt, and writing to a table does not shrink the window within a session. Journaling is not a within-session workaround; it is what makes ending a session survivable. The agent dies, the journal persists, the next agent hydrates from a small slice. Context stays small because the journal's read API does the filtering, not the model. Agents are cheap and replaceable; the journal is durable.

Gates (autonomous mode). The journal pairs with the two execution modes: interactive (Travis approves each advance) and autonomous (Lovebug completes + documents, stops at hard gates). Gate categories worth encoding — operations where cost-of-wrong is high and cost-of-asking is low: outside the project tree (existing), sudo/privilege escalation, network-facing config (firewall/proxy/DNS), secrets touched, destructive data ops, git ops beyond the working branch, dependency/package additions, cross-service changes. The journal should log gate events ("considered touching X outside project, stayed in-scope") as evidence the gate logic fired — not just that it never came up.

Decision-doc contract for autonomous mode. When Lovebug makes unsupervised choices, the handoff/notes must be auditable without re-reading transcripts: what was the decision point (options on the table), which option, why (rule-out for the rest), what would have triggered a gate.

Prior art. External searches confirm the pattern is established (blackboard / persistent governed context layer that agents read and write; downstream agents query the layer rather than upstream transient session memory). What is not out there is a purpose-built, plan-keyed execution journal as a first-class orchestration participant for a personal stack — existing implementations are either heavyweight frameworks, generic primitives (Redis/vector DBs), or audit/compliance tooling meant for regulators rather than for the orchestrator to read back. This is the right shape for the herbylab stack instead of adopting someone else's framework.

Relationship to spec-driven workflow. Task-ID addressing nudges plans toward discrete, addressable, spec-shaped tasks — the same direction the spec-driven / plan-interviewer work is already heading. Aligned, not additional.

9.9 KiB Raw Blame History Unescape Escape