--- type: synthesis path: Wiki/Dev tags: - ob1 - embeddings - pgvector - wiki - automation - mcp sources: - "[[Sources/Dev/ai-memory-architecture-research]]" - "[[Sources/Dev/second-brain]]" - "[[Sources/Homelab/ob1-main-deployment]]" - "[[Sources/Homelab/ob1-deployment]]" - "[[Sources/Dev/email-to-obsidian-automation]]" - "[[Sources/Dev/email-obsidian-git-pipeline]]" - "[[Sources/Homelab/homelab-mcp-server]]" - "[[Sources/Homelab/2026-05-06-homelab-mcp-server-stand-up]]" - "[[Sources/Dev/wiki-vault-migration]]" created: 2026-05-08 updated: 2026-05-08 --- # AI Memory Architecture ## Synthesis Travis's AI memory system is not a single tool but a layered architecture: a structured wiki vault for human-readable knowledge, a pgvector semantic store for AI-queryable memory, a homelab MCP server to expose both to AI agents, and n8n pipelines to keep all layers in sync. These components appear across 8+ projects and represent a coherent long-term investment in AI-assisted knowledge work. ### The Three Memory Layers ``` Layer 1: Structured Wiki (wiki-vault) - Human-readable Markdown - Obsidian wikilinks + navigation - Compiled by LLM (wiki-maintenance skill) - Git-versioned - Written by: MCP server, email pipeline, Travis/Jenny Layer 2: Semantic Memory (OB1) - Supabase + pgvector - Embeddings of vault content + session context - Queryable by AI agents via semantic search - Written by: n8n parallel pipeline (mirrors vault writes) - Access: Tailscale-only Layer 3: Agent Context (Dispatch / Lovebug) - Active working memory within Claude Code context windows - Pre-compact hook (Phase 7) will drain important context → OB1 - MCP server provides tool access to query OB1 and wiki from active sessions ``` ### Ingest Paths Multiple input channels feed into the wiki-vault and, in parallel, into OB1: 1. **MCP server** (`mcp.herbylab.dev`) — Claude Code and other AI agents write session notes and project updates directly to `Sources/` via the MCP tools. This is the primary agentic write path. 2. **Email pipeline** — Gmail → n8n → Gitea push → vault. Designed for mobile/async capture: send an email, it becomes a session note. Built and validated in the email-obsidian-git-pipeline session. 3. **Manual / direct** — Travis or Jenny write directly to the vault via Obsidian. 4. **Parallel OB1 write** (planned) — n8n intercepts all vault writes and mirrors them to OB1 as embeddings. Not yet live; pending OB1 deployment Phase 4. ### Query Paths - **Wiki** — humans browse via Obsidian; AI agents query via MCP vault-read tools - **OB1** — AI agents query via MCP OB1-recall tools (semantic search over embeddings) - **Context injection** — the homelab MCP server exposes both layers as Claude Code tools, enabling Lovebug to pull relevant wiki pages or OB1 memories as needed within a session ### Harness vs. Model Architecture The AI memory research session explored the design question: should AI memory be implemented at the **model layer** (fine-tuning, RLHF on personal data) or the **harness layer** (RAG, context injection, external retrieval)? The conclusion was harness-layer: model weights are slow and expensive to update, while a pgvector store + MCP injection is fast, reversible, and inspectable. Poolside and Pi.dev were evaluated as examples of model-layer approaches — useful for code but not for personal episodic memory. The two memory modes are distinct: - **Episodic** (session notes, project logs) → OB1 embeddings for temporal/event recall - **Semantic** (wiki entity pages, synthesis) → wiki-vault for stable structured knowledge ### The Second-Brain Pattern The `second-brain` project captures the Karpathy LLM wiki pattern: pull raw content → transcribe → extract entities/facts → review → publish to wiki. This is exactly what the wiki-maintenance compile loop implements. The compile loop is the "review → publish" step; the MCP server + email pipeline are the "pull → capture" steps. OB1 adds the AI-queryable retrieval layer that the pure wiki-markdown model lacks. ### MCP Server as the Integration Point The homelab MCP server (`mcp.herbylab.dev`, deployed 2026-05-06) is the critical integration point that makes this architecture operational. Before MCP, each layer was separate — the wiki was browsed in Obsidian, OB1 was planned but not integrated, and AI agents had no way to query either during a session. The MCP server exposes vault tools and (once OB1 is live) OB1 recall as Claude Code tool calls, making the entire memory stack available to Lovebug mid-session. ## Sources **ai-memory-architecture-research** (session notes) provides the harness-vs-model architecture decision, the pgvector choice, and the Poolside/Pi.dev research. This is the foundational design session for OB1. **second-brain** (project plan) captures the Karpathy LLM wiki pattern and maps it to the wiki-vault compile loop + MCP server design. Establishes the pull→transcribe→extract→review→publish pipeline. **ob1-main-deployment** (project plan) provides the full OB1 deployment architecture: Supabase + pgvector, Phases 0–6, embedding pipeline design. **ob1-deployment** (addendum) captures the locked decisions: Tailscale-only access, n8n parallel write pipeline, Phase 7 Dispatch pre-compact hook. Critical refinements to the main plan. **email-to-obsidian-automation** and **email-obsidian-git-pipeline** document the Gmail → n8n → Gitea → vault ingest path. The pipeline is built and validated as of May 2026. **homelab-mcp-server** and **2026-05-06-homelab-mcp-server-stand-up** document the MCP server deployment: FastMCP Streamable HTTP, Authentik OAuth2, Cloudflare Tunnel. The integration point connecting all memory layers to AI agents. **wiki-vault-migration** documents the vault consolidation from pbs-projects + homelab-projects into this unified structure, establishing the wiki-vault as the canonical structured knowledge store. ## Open questions - OB1 Phases 0–6 deployment timeline: what is the current blocker? The addendum was written but the deployment plan doesn't indicate Phase 0 is complete. - When the n8n parallel write pipeline goes live, will it backfill existing vault content (all 62+ notes) into OB1, or only new writes going forward? - The Dispatch pre-compact hook (Phase 7) depends on Dispatch exposing a lifecycle event. Is this a Dispatch feature that already exists, or does it require Dispatch development work? - How will the MCP server's OB1 recall tool handle stale embeddings when vault content is updated by the compile loop? Will there be a re-embed step in the compile procedure? - The wiki + OB1 parallel write model creates two sources of truth for the same content. Reconciliation strategy (wiki as authoritative; OB1 as derived index) should be documented once the pipeline goes live.