diff --git a/Tech/Projects/ob1-deployment.md b/Tech/Projects/ob1-deployment.md index 0ea5990..ba28eb2 100644 --- a/Tech/Projects/ob1-deployment.md +++ b/Tech/Projects/ob1-deployment.md @@ -10,248 +10,278 @@ tags: - ob1 - supabase - mcp - - docker + - addendum created: 2026-05-04 updated: 2026-05-04 --- -# OB1 Deployment Project Plan +# OB1 Deployment — Addendum -Deploy OpenBrain (OB1) as the personal AI memory layer on the homelab. -OB1 is a self-hosted Postgres-plus-Supabase stack exposing MCP server -access for any compatible AI client. This becomes the persistent -memory layer underneath Claude Code Dispatch, Claude Desktop, and -eventually Pi or other harnesses. +This addendum amends the original `ob1-deployment` project plan +based on architectural decisions made during the wiki-construct +design session on 2026-05-04. It does not replace the original +plan; it modifies specific phases and clarifies scope. Read this +alongside the original plan. -This plan is structured for hand-off to an agent after the OB1 repo is -cloned. Each phase has explicit deliverables, validation steps, and -stop conditions. The agent should pause after each phase for review -before proceeding. +## Context for the changes -## Background context the agent needs +The wiki-construct session crystallized the framing for personal +AI memory across the homelab: -- OB1 source: `github.com/NateBJones-Projects/OB1` -- Architecture: Supabase (Postgres + GoTrue + PostgREST + Kong + -Studio + Edge Functions) with OB1's MCP server layered on top -- Target deployment node: Proxmox VM on the existing homelab (assume a -fresh Ubuntu 24.04 LTS VM unless directed otherwise) -- Network: Tailscale mesh for remote access; Traefik already running -on the homelab as the reverse proxy with HTTPS -- Auth model: Authelia in front of web-facing surfaces (Studio UI); -GoTrue stays for application identity and JWT issuance for AI clients -- This is a personal single-user deployment initially. RLS policies -should be in place but expect future expansion to a second user -(Jenny) for shared PBS context -- The user prefers Ansible for deployments and Docker Compose for -service orchestration. UV for any Python work. Never use ---break-system-packages +- **Wiki** is Travis's interface to his own thinking. Human- + facing, organized for browsing in Obsidian. Agents read it + only when explicitly pointed at a file. +- **OB1** is Lovebug's working memory. Agent-facing, semantic + search via MCP, captured automatically from sessions and + conversations. -## Phase 0 — Repository inspection and environment audit +The two systems are parallel and complementary, not unified. +This framing changes the OB1 plan less than expected — most of +the original plan still stands — but it sharpens scope around +what OB1 is and isn't responsible for. -Goal: understand what the agent is actually deploying before touching anything. +## Amendments to the original plan -Tasks: -- [ ] Read the OB1 README and any deployment or setup docs in the repo -- [ ] Inventory the docker-compose.yml — list every service, its -image, ports, volumes, dependencies -- [ ] Identify which Supabase services OB1 actually requires versus -which are bundled but unused (Realtime, Storage, ImgProxy, Logflare -are likely candidates for disabling) -- [ ] Check the OB1 MCP server implementation — language, runtime, -dependencies, how it connects to Supabase -- [ ] Note any environment variables, secrets, or external -dependencies the deployment requires -- [ ] Check schema or migration files — understand the data model -before touching it +### Phase 4 — Authelia and Traefik (deferred) -Deliverable: a written summary of what OB1 deploys, what's optional, -what's required, and what external resources (API keys, URLs, secrets) -the user needs to provide. +The original plan put Authelia + Traefik fronting in Phase 4 as +part of baseline deployment. **Defer this entire phase** until +there is a concrete user-facing reason to expose OB1 outside +Tailscale. -Stop condition: present the summary and the proposed Phase 1 plan to -the user for review before proceeding. +Replacement scope for Phase 4: -## Phase 1 — VM provisioning and base setup +- Tailscale-only access for all OB1 surfaces (Studio UI, REST + API, MCP server) +- Document the URL map for the user (Tailscale hostnames / + ports for each surface) +- Confirm that the JWT-bearing MCP path stays Tailscale-only + forever — even when Authelia is eventually added in front of + browser surfaces, agent endpoints stay on Tailscale. This is + a design invariant, not a phase choice. -Goal: clean Ubuntu 24.04 VM ready to run OB1. +Authelia goes back on the roadmap when Jenny's interface lands +or when there's another reason to put a browser surface on the +public web. Not before. -Tasks: -- [ ] Confirm with user: VM hostname, target Proxmox node, resource -allocation (suggest 4 vCPU, 8GB RAM, 60GB disk as starting point — -adjust based on Phase 0 findings) -- [ ] Provision the VM via Proxmox (the user can do this manually; -agent should provide exact specs) -- [ ] Install Docker Engine and Docker Compose v2 (not the deprecated -docker-compose v1) -- [ ] Install Tailscale and join the mesh -- [ ] Confirm Tailscale connectivity from the user's dev rig -- [ ] Set up a non-root user for service operation (suggest -`ob1admin`) with Docker group membership -- [ ] Install basic operational tooling: htop, ncdu, git, curl, jq +This shortens the deployment by an estimated 30-40% of the +Phase 4 task list and removes the most complex coordination +with the existing Traefik / Authelia config. -Deliverable: a working VM accessible over Tailscale, with Docker -installed and the OB1 repo cloned to `/opt/ob1/` (or wherever the user -prefers). +### Phase 3 — schema decisions (extended) -Stop condition: confirm with user that the VM is reachable and base -setup is complete before proceeding to Phase 2. +The original Phase 3 deploys OB1's schema as-shipped. **Add one +forward-looking schema decision** before applying migrations: -## Phase 2 — Trimmed Supabase deployment +- **User scoping from day one.** Schema includes a `user_id` or + equivalent tenant column on every captured row, with RLS + policies enforced, even though the deployment is single-user + (Travis only) at first. -Goal: get the Supabase stack running with only the services OB1 actually uses. +Rationale: Jenny's eventual interface (PBS-public web surface +served from the PBS site) will require multi-user support. Row- +level security retrofitted later is painful. RLS designed in +from day one is cheap. -Tasks: -- [ ] Based on Phase 0 inventory, modify docker-compose.yml to disable -services OB1 doesn't require. Likely candidates for removal: Realtime, -Storage, ImgProxy, Logflare, Vector -- [ ] Generate strong secrets for: Postgres password, JWT secret, anon -and service role keys, dashboard credentials. Use the Supabase secret -generation pattern; do not reuse defaults -- [ ] Configure Postgres with conservative resource settings: -max_connections=20, shared_buffers=256MB. These can be tuned upward -later if needed -- [ ] Configure Kong's routing to only expose the services that remain -- [ ] Bring up the stack with `docker compose up -d` and wait for all -services to report healthy -- [ ] Verify Postgres is reachable from inside the Docker network -- [ ] Verify GoTrue is issuing tokens by creating a test user via the auth API -- [ ] Verify PostgREST is serving the auto-generated REST API -- [ ] Access Supabase Studio UI via Tailscale and confirm it loads +This is a "design choice, not a feature" addition — no UI for +multi-user, no actual second user yet, just schema and policies +in place so the door isn't accidentally closed. -Deliverable: a running, trimmed Supabase stack on the VM. All services -healthy. Studio UI reachable over Tailscale. +### Phase 5 — client integration (refined) -Stop condition: confirm with user that Supabase is up and Studio is -accessible. Get approval before proceeding to Phase 3. +The original Phase 5 wires Claude Code, Claude Desktop, and the +n8n markdown-summary flow to OB1. All three remain in scope. +**Refinement:** the n8n flow modification should write to OB1 +*in parallel* with the existing Obsidian write, not in place of +it. The two systems serve different purposes: -## Phase 3 — OB1 schema and MCP server +- The Obsidian write builds Travis's wiki (now structured per + the wiki-construct project) +- The OB1 write builds Lovebug's memory -Goal: deploy OB1's database schema and MCP server on top of the -working Supabase stack. +Same source, two destinations, neither replacing the other. -Tasks: -- [ ] Apply OB1's database migrations to Postgres — create the -thoughts table, any related tables, RLS policies, indexes including -the pgvector HNSW index -- [ ] Configure OB1's MCP server with the Supabase connection details, -JWT secret, and any API keys it needs (OpenAI for embeddings, or -Ollama URL for local embeddings — user preference) -- [ ] Decide with the user: cloud embeddings (OpenAI) for higher -quality, or local embeddings (Ollama on the dev rig with the 3080) for -zero cost and full data locality. Recommend local embeddings given -user's homelab philosophy and existing Ollama setup -- [ ] Run the MCP server (likely as a Docker container alongside -Supabase, or as a systemd service depending on what OB1 ships) -- [ ] Verify the MCP server starts cleanly and connects to Postgres -- [ ] Test capture: insert a test thought via the MCP server's capture -endpoint and verify it lands in the thoughts table with embedding and -metadata populated -- [ ] Test retrieval: query the MCP server's search endpoint and -verify it returns the test thought via semantic search +The n8n node added in Phase 5 should: -Deliverable: OB1 MCP server running and operational. Test thought -captured and retrievable. +- POST the captured content to OB1's REST API +- Map relevant frontmatter fields to OB1 metadata (project, + domain, tags, dates) +- Tag OB1 captures with their wiki-vault path so cross- + reference is possible later if needed +- Run after the Obsidian write succeeds (Obsidian is the + primary destination; OB1 capture is additive) -Stop condition: confirm with user that capture and retrieval work -end-to-end before proceeding. +### Phase 6 — operational hygiene (no changes) -## Phase 4 — Authelia integration and Traefik routing +The original Phase 6 stands. Backups, monitoring, Ansible +playbook, runbook, snapshot. Unchanged. -Goal: put OB1 behind the homelab's existing Authelia/Traefik perimeter. +### Phase 7 — Dispatch pre-compact hook (promoted from out-of-scope) -Tasks: -- [ ] Add Traefik labels (or update Traefik dynamic config) to route to: - - Supabase Studio UI (Authelia-protected) - - PostgREST API (Authelia-protected for direct access; bypassed for -MCP server which uses JWTs) - - GoTrue auth endpoints (no Authelia — these need to be reachable -for JWT operations) - - OB1 MCP server endpoint (no Authelia — clients use JWTs, not -browser sessions) -- [ ] Configure Authelia access control rules for the new routes -- [ ] Verify HTTPS works for all exposed endpoints via Traefik -- [ ] Test browser access to Studio: should hit Authelia login, then -proceed to Studio -- [ ] Test MCP server access: should bypass Authelia and authenticate -via JWT only -- [ ] Document the final URL map for the user +The original plan listed the Dispatch pre-compact hook as out- +of-scope and "separate project". **Promote this to Phase 7 of +this project**, immediately following Phase 6, as the closing +phase. -Deliverable: OB1 properly fronted by Traefik with Authelia gating -browser surfaces. JWT-based clients reach the MCP server directly. +Reason: until the pre-compact hook is shipping captures into +OB1, OB1 is not actually serving its primary purpose as +Lovebug's working memory. Phases 1-6 deploy a database with +Claude Code MCP captures and email-summary captures, but no +Dispatch session captures. Dispatch sessions are where the bulk +of Lovebug's work happens. Without the hook, OB1 has the wrong +memory. -Stop condition: confirm full access pattern works from both browser -(Authelia path) and AI client (JWT path). +Phase 7 scope: -## Phase 5 — Client integration +- Design what Dispatch sessions capture (full transcript, + decisions only, summaries only — needs design work) +- Design noise filtering (Dispatch sessions include a lot of + routine command output that shouldn't end up in semantic + memory) +- Implement the pre-compact hook integration +- Test capture end-to-end: run a Dispatch session, verify + meaningful content lands in OB1, verify retrieval surfaces + the right thing +- Document the hook for future modification -Goal: connect Claude Code, Claude Desktop, and the existing n8n flows to OB1. +Phase 7 is its own meaningful design exercise — what to capture +is not obvious. Treat it as a multi-session phase with its own +design step before implementation. -Tasks: -- [ ] Generate a long-lived JWT or service-role API token for AI -client use. Store in the user's KeePassXC database -- [ ] Configure Claude Code's MCP integration to point at OB1's MCP -server. Test capture and search from a Claude Code session -- [ ] Configure Claude Desktop's MCP integration similarly. Test -capture and search from a Desktop conversation -- [ ] Add a node to the existing n8n markdown-summary intake flow that -POSTs captured content to OB1's REST API in parallel with the existing -Obsidian write. Map frontmatter fields to OB1 metadata -- [ ] Test the full capture loop: send a test summary email through -the existing flow and verify it lands in both Obsidian and OB1 -- [ ] Document the n8n flow modifications for the user +## Scope clarifications (no plan change, just sharpening) -Deliverable: OB1 receiving captures from Claude Code, Claude Desktop, -and the n8n email-intake pipeline. All three paths verified working. +### What OB1 is for -Stop condition: confirm all three capture paths work and the user can -perform an end-to-end test from any AI client. +- Lovebug's working memory across Dispatch sessions +- Lovebug's memory across Claude Code sessions +- Captures of Travis's email-summary intake (the same content + that builds the wiki, captured in parallel for semantic + search) +- Eventually: agent-execution session notes from coding + projects (currently captured alongside code in the project + repo; eventually flow into OB1 for cross-project recall) -## Phase 6 — Operational hygiene +### What OB1 is not for -Goal: make the deployment maintainable. +- Travis's curated knowledge (that's the wiki) +- Long-form synthesis (that's the wiki) +- Human browsing (Studio UI is for inspection and + troubleshooting, not daily reference) +- Replacement for the wiki — the two systems coexist -Tasks: -- [ ] Set up a daily Postgres backup via pg_dump to the Ubuntu NAS -over Tailscale -- [ ] Add the OB1 VM to existing monitoring (Uptime Kuma if it's -monitoring infra services) -- [ ] Document the deploy in an Ansible playbook so future -re-provisioning is automated -- [ ] Write a brief runbook covering: how to update OB1, how to -restore from backup, how to rotate the JWT secret, how to add a new -client -- [ ] Snapshot the VM via Proxmox once everything is verified working +### How OB1 and the wiki interact -Deliverable: backups running, monitoring in place, Ansible playbook -captures the deploy, runbook written, snapshot taken. +In Phase 1 of both projects: they don't, beyond the parallel +n8n write described above. Travis uses the wiki to think. +Lovebug uses OB1 to remember its own work and conversations. -Stop condition: project complete. Hand off to user for ongoing use. +In some Phase 2 future: a context assembler may query both +layers when bundling prompts — semantic search across OB1 plus +explicit wiki references when Travis points at specific pages. +This is deferred and should not influence Phase 1 architecture +decisions. -## Out of scope +The wiki should never be designed for OB1 to consume, and OB1 +should never be designed to maintain wiki content. Keep them +parallel. -These are intentionally not part of this plan: +## Dependencies and ordering -- Building a custom technical-project extension for OB1 (separate -project, after baseline is operational) -- Adding Jenny as a second user with PBS-scoped access (separate -project, after Jenny's content workflow lands) -- Pre-compact hook for Dispatch transcript capture (separate project — -needs design work on what to capture and how to filter noise) -- Migrating existing Obsidian content into OB1 retroactively (separate -project — bulk import path) +The wiki-construct project does not block OB1. They can run in +parallel. Travis's stated preference is wiki-first, but that's a +bandwidth decision, not a technical dependency. -## Agent operating instructions +If wiki-construct ships first: -- Work one phase at a time. Stop and report at the end of each phase. -Wait for user confirmation before proceeding to the next phase -- For any ambiguous decision, ask the user rather than guessing -- Never expose the homelab to the public internet during this -deployment. Tailscale mesh is the only access path -- All secrets generated during deployment go into the user's KeePassXC -database. Never write them to disk in plaintext outside the running -Docker environment -- Use the user's preferred tooling: Docker Compose for orchestration, -Ansible for repeatable deployment steps, UV for any Python work -- If a step fails, do not retry blindly. Report the failure with the -actual error output and propose next steps -- The user prefers concise updates — report what was done, what's -next, and any blockers. Skip the celebratory framing \ No newline at end of file +- The n8n markdown-summary flow will already be writing to a + structured wiki when OB1 Phase 5 adds the parallel OB1 write +- Frontmatter conventions will already be stable, simplifying + the OB1 metadata mapping + +If OB1 ships first: + +- Wiki-construct's compile loop is unaffected (it doesn't read + from OB1) +- The Phase 5 n8n write will need to be revisited once the + wiki's frontmatter conventions land + +Either order works. + +## Open items + +- [ ] Decide cloud (OpenAI) vs. local (Ollama on RTX 3080) + embeddings during Phase 3 (recommendation in original plan + stands: local) +- [ ] Phase 7 design: what to capture from Dispatch sessions and + how to filter noise +- [ ] Confirm OB1's schema supports `user_id` / tenant scoping + cleanly during Phase 0 inventory; if not, this becomes a + schema extension rather than a configuration choice + +## Tasks (additions and changes only) + +### Phase 0 — Repository inspection (one addition) + +- [ ] Confirm OB1's schema supports user-scoping or determine + the minimal extension needed + +### Phase 3 — Schema and MCP server (one addition) + +- [ ] Apply user-scoping schema extension if needed +- [ ] Verify RLS policies enforce single-user access correctly + (Travis sees only Travis's rows, even though there's only + one user) + +### Phase 4 — Tailscale-only access (replaces original Phase 4) + +- [ ] Document Tailscale URL map for all OB1 surfaces +- [ ] Confirm MCP path is reachable over Tailscale from Travis's + dev rig and from herbydev +- [ ] Document the design invariant: MCP path stays Tailscale- + only permanently +- [ ] Defer Authelia + Traefik integration explicitly + +### Phase 5 — Client integration (one refinement) + +- [ ] Confirm n8n write to OB1 runs in parallel with Obsidian + write, not in place of it +- [ ] Tag OB1 captures from the email flow with their wiki-vault + path + +### Phase 7 — Dispatch pre-compact hook (new) + +- [ ] Design: what Dispatch sessions capture +- [ ] Design: noise filtering rules +- [ ] Implementation: pre-compact hook integration +- [ ] Test: end-to-end capture and retrieval from a Dispatch + session +- [ ] Document: hook configuration for future modification + +## Phase 1 done when (amended) + +The original plan's "project complete" trigger stands, with one +addition: **the Dispatch pre-compact hook is shipping captures +into OB1 and Lovebug is using them.** Without that, OB1 is +deployed but not yet doing its primary job. + +Consider OB1 truly operational only after Phase 7. Phases 1-6 +deploy the substrate; Phase 7 turns it on. + +## Notes + +- This addendum is not a re-plan. The original phases 1-3 and + 6 are unchanged. Phase 4 is shrunk. Phase 5 is refined. + Phase 7 is added. +- The wiki-construct project is the sibling project. Its plan + and skill bundle land separately. They do not depend on each + other. +- The framing decisions in this addendum (wiki for human, OB1 + for agent, parallel not unified) are load-bearing. If + anything in the deployment seems to push back on that + framing — e.g., "should OB1 also serve the wiki?" or "should + the wiki feed OB1 automatically?" — stop and confirm with + Travis before proceeding. The instinct to unify is wrong; + the instinct to keep parallel is correct. + + +-- +...sent from Jenny & Travis \ No newline at end of file