--- type: synthesis path: Wiki/Dev tags: - ob1 - embeddings - pgvector - wiki - automation - mcp sources: - "[[Sources/Dev/ai-memory-architecture-research]]" - "[[Sources/Dev/second-brain]]" - "[[Sources/Homelab/ob1-main-deployment]]" - "[[Sources/Homelab/ob1-deployment]]" - "[[Sources/Dev/email-to-obsidian-automation]]" - "[[Sources/Dev/email-obsidian-git-pipeline]]" - "[[Sources/Homelab/homelab-mcp-server]]" - "[[Sources/Homelab/2026-05-06-homelab-mcp-server-stand-up]]" - "[[Sources/Dev/wiki-vault-migration]]" - "[[Sources/Homelab/postgres-consolidation-ob1-reflection-layer]]" - "[[Sources/Homelab/postgres-consolidation-followups]]" - "[[Sources/Homelab/2026-05-15-postgres-consolidation-phase0-recon]]" - "[[Sources/Homelab/2026-05-15-postgres-consolidation-phase2a-result]]" - "[[Sources/Homelab/ob1-jsonl-watcher-ambient-capture-from-claude-code-sessions|ob1-jsonl-watcher]]" - "[[Sources/Homelab/vault-semantic-search]]" - "[[Sources/Homelab/herbylab-project-management]]" - "[[Sources/Homelab/trellis-mcp]]" created: 2026-05-08 updated: 2026-05-24 --- # AI Memory Architecture ## Synthesis Travis's AI memory system is not a single tool but a layered architecture: a structured wiki vault for human-readable knowledge, a pgvector semantic store for AI-queryable memory, a homelab MCP server to expose both to AI agents, and pipelines to keep all layers in sync. These components appear across 15+ projects and represent a coherent long-term investment in AI-assisted knowledge work. > **Update (2026-05-24 compile).** Two things in the original synthesis below are now superseded and corrected in-line: (1) OB1 was never "Supabase" — it's a self-hosted `pgvector/pgvector:pg17` container, since consolidated into the `petalbrain` database; (2) the "n8n parallel write" ingest path for OB1 was replaced by a JSONL session watcher. The **Consolidated Substrate** and **Trellis: the third leg** sections below capture the May-2026 state; the older sections are kept for design-history continuity. ### The Consolidated Substrate (`petalbrain`) As of the May-2026 postgres consolidation, all three memory systems share one Postgres instance (`homelab-postgres`, pgvector:pg17) under one database, **`petalbrain`**, with schema-level isolation and a dedicated service role per system: ``` petalbrain (Postgres / pgvector:pg17, on Proxmox, Tailscale-only) ├── schema wiki — owner vault_mcp — wiki.artifacts / projects / notes ├── schema ob1 — owner ob1_mcp — ob1.thoughts (raw) + ob1.reflections (compiled) ├── schema trellis — owner trellis_mcp — threads / events / tags └── public.embeddings — shared vector store, all sources, 768-d nomic-embed-text HNSW vector_cosine_ops (m=16, ef_construction=64) 512-token chunks / 64-token overlap ``` `public.embeddings` is the load-bearing idea: one table for every source, keyed by `(source_schema, source_table, source_id, embedding_model)`, so "find similar to X across everything" is a single `ORDER BY embedding <-> query` rather than a UNION across per-system tables. Cross-schema reads are granted narrowly (ob1_mcp reads wiki + trellis for reflection synthesis; lovebug is RW everywhere). Isolation is enforced by Postgres at the role level, not by trusting MCP code to behave — matching the same "separate the *who*, not the *where*" pattern used for host users and Authentik. The embedding model is **local Ollama `nomic-embed-text` (768-d)** — never a hosted provider for memory. Switching it is a deliberate full re-embed, which is why `embedding_model` is part of the uniqueness key. ### OB1's own two layers OB1 now mirrors the wiki's Sources/Wiki split internally: `ob1.thoughts` is the raw capture layer; `ob1.reflections` is a compiled layer written by a **daily reflection job** (`claude-sonnet-4-6` via the Anthropic Messages API direct) that synthesizes generalizable lessons with citations back to source rows. Same discipline as the wiki — mixing raw and synthesized content in one table pollutes semantic neighbors. Ambient capture is the [[Sources/Homelab/ob1-jsonl-watcher-ambient-capture-from-claude-code-sessions|JSONL watcher]]: it tails Claude Code / Cowork session files and posts turn-pairs to OB1's `capture_thought`. "Save everything, filter later." ### Trellis: the third leg The original two-system framing (wiki + OB1) is now three: [[Sources/Homelab/trellis-mcp|Trellis]] is resumable *working state* — named threads with checkpoints that survive conversation boundaries. The clean separation the stack settled on: **CLAUDE.md = instructions (how to behave), Trellis = record (where we are in the doing), wiki + OB1 = lessons and outcomes (what we learned/built).** None should swallow another's job. Trellis lives in `petalbrain.trellis`; its tool surface is being refined ([[Sources/Homelab/trellis-mcp-tool-surface-followups|checkpoint_thread rename + id addressing]]). ### The Three Memory Layers ``` Layer 1: Structured Wiki (wiki-vault) - Human-readable Markdown - Obsidian wikilinks + navigation - Compiled by LLM (wiki-maintenance skill) - Git-versioned - Written by: MCP server, email pipeline, Travis/Jenny Layer 2: Semantic Memory (OB1) [see "Consolidated Substrate" above for current state] - self-hosted pgvector:pg17 (petalbrain), NOT Supabase - Embeddings of vault content + session context (Ollama nomic-embed-text, 768-d) - Queryable by AI agents via semantic search over public.embeddings - Written by: JSONL session watcher → capture_thought (NOT n8n parallel write) - Access: Tailscale-only Layer 3: Agent Context (Dispatch / Lovebug) - Active working memory within Claude Code context windows - Pre-compact hook (Phase 7) will drain important context → OB1 - MCP server provides tool access to query OB1 and wiki from active sessions ``` ### Ingest Paths Multiple input channels feed into the wiki-vault and, in parallel, into OB1: 1. **MCP server** (`mcp.herbylab.dev`) — Claude Code and other AI agents write session notes and project updates directly to `Sources/` via the MCP tools. This is the primary agentic write path. 2. **Email pipeline** — Gmail → n8n → Gitea push → vault. Designed for mobile/async capture: send an email, it becomes a session note. Built and validated in the email-obsidian-git-pipeline session. 3. **Manual / direct** — Travis or Jenny write directly to the vault via Obsidian. 4. **Parallel OB1 write** (planned) — n8n intercepts all vault writes and mirrors them to OB1 as embeddings. Not yet live; pending OB1 deployment Phase 4. ### Query Paths - **Wiki** — humans browse via Obsidian; AI agents query via MCP vault-read tools - **OB1** — AI agents query via MCP OB1-recall tools (semantic search over embeddings) - **Context injection** — the homelab MCP server exposes both layers as Claude Code tools, enabling Lovebug to pull relevant wiki pages or OB1 memories as needed within a session ### Harness vs. Model Architecture The AI memory research session explored the design question: should AI memory be implemented at the **model layer** (fine-tuning, RLHF on personal data) or the **harness layer** (RAG, context injection, external retrieval)? The conclusion was harness-layer: model weights are slow and expensive to update, while a pgvector store + MCP injection is fast, reversible, and inspectable. Poolside and Pi.dev were evaluated as examples of model-layer approaches — useful for code but not for personal episodic memory. The two memory modes are distinct: - **Episodic** (session notes, project logs) → OB1 embeddings for temporal/event recall - **Semantic** (wiki entity pages, synthesis) → wiki-vault for stable structured knowledge ### The Second-Brain Pattern The `second-brain` project captures the Karpathy LLM wiki pattern: pull raw content → transcribe → extract entities/facts → review → publish to wiki. This is exactly what the wiki-maintenance compile loop implements. The compile loop is the "review → publish" step; the MCP server + email pipeline are the "pull → capture" steps. OB1 adds the AI-queryable retrieval layer that the pure wiki-markdown model lacks. ### MCP Server as the Integration Point The homelab MCP server (`mcp.herbylab.dev`, deployed 2026-05-06) is the critical integration point that makes this architecture operational. Before MCP, each layer was separate — the wiki was browsed in Obsidian, OB1 was planned but not integrated, and AI agents had no way to query either during a session. The MCP server exposes vault tools and (once OB1 is live) OB1 recall as Claude Code tool calls, making the entire memory stack available to Lovebug mid-session. ## Sources **ai-memory-architecture-research** (session notes) provides the harness-vs-model architecture decision, the pgvector choice, and the Poolside/Pi.dev research. This is the foundational design session for OB1. **second-brain** (project plan) captures the Karpathy LLM wiki pattern and maps it to the wiki-vault compile loop + MCP server design. Establishes the pull→transcribe→extract→review→publish pipeline. **ob1-main-deployment** (project plan) provides the full OB1 deployment architecture: Supabase + pgvector, Phases 0–6, embedding pipeline design. **ob1-deployment** (addendum) captures the locked decisions: Tailscale-only access, n8n parallel write pipeline, Phase 7 Dispatch pre-compact hook. Critical refinements to the main plan. **email-to-obsidian-automation** and **email-obsidian-git-pipeline** document the Gmail → n8n → Gitea → vault ingest path. The pipeline is built and validated as of May 2026. **homelab-mcp-server** and **2026-05-06-homelab-mcp-server-stand-up** document the MCP server deployment: FastMCP Streamable HTTP, Authentik OAuth2, Cloudflare Tunnel. The integration point connecting all memory layers to AI agents. **wiki-vault-migration** documents the vault consolidation from pbs-projects + homelab-projects into this unified structure, establishing the wiki-vault as the canonical structured knowledge store. **postgres-consolidation-ob1-reflection-layer** (project plan) is the source for the single-database / three-schema / four-service-user model, the shared `public.embeddings` design, and OB1's two-layer captures+reflections split. **2026-05-15-postgres-consolidation-phase0-recon** is the primary evidence that OB1 is pgvector:pg17 + Ollama `nomic-embed-text` (768-d), not Supabase — it inventories the live instance directly. **phase2a-result** confirms the schemas, roles, and `public.embeddings` table landed with verified permission isolation. **postgres-consolidation-followups** supplies the reflection-model choice (`claude-sonnet-4-6` direct), chunking parameters (512/64), and the enricher loop bug. **ob1-jsonl-watcher** (project plan) is the source for the ambient-capture-via-JSONL design that replaced the n8n parallel-write idea. **trellis-mcp** establishes the third memory leg and the instructions/record/lessons three-way separation. **herbylab-project-management** documents the Postgres-backed artifact model and the NOTIFY→n8n→Trello projection. **vault-semantic-search** specifies the (still-unbuilt) `vault:search` tool over the wiki schema. ## Open questions *Several earlier open questions were resolved by the May-2026 consolidation: OB1's substrate is confirmed (pgvector:pg17 / `petalbrain`, not Supabase); ambient capture is the JSONL watcher, not n8n parallel-write (so there is no wiki↔OB1 dual-source-of-truth problem — vault is content, OB1 indexes session content); and the shared `public.embeddings` table is live.* Still open: - **Reflector quality** — the daily reflection job's synthesis prompt is unproven; first non-dry-run output needs review before the 3am timer is trusted (postgres-consolidation-followups Phase 1). - **`ob1-enricher` PENDING_QUERY loop** — must be fixed before the enricher is restarted, or the metadata backlog grows. - **Re-embed on compile** — when the wiki compile loop edits a Wiki page, its `public.embeddings` row goes stale. There is still no re-embed step in the compile procedure. *(Agent inference, not stated in any source — flagged.)* - **`vault:search` not yet built** — the semantic-search MCP tool over the wiki schema is designed ([[Sources/Homelab/vault-semantic-search|vault-semantic-search]]) but unbuilt; until it ships, the wiki half of the substrate isn't actually queryable by agents. - **claude/competent-knuth-de9483 branch** — artifact tooling that landed via a separate path; merge/abandon/cherry-pick still undecided. - **Dispatch pre-compact hook (Phase 7)** — still depends on a Dispatch lifecycle event; the JSONL watcher covers Claude Code/Cowork capture in the meantime.