wiki-vault/Wiki/Dev/ai-memory-architecture.md

---
type: synthesis
path: Wiki/Dev
tags:
  - ob1
  - embeddings
  - pgvector
  - wiki
  - automation
  - mcp
sources:
  - "[[Sources/Dev/ai-memory-architecture-research]]"
  - "[[Sources/Dev/second-brain]]"
  - "[[Sources/Homelab/ob1-main-deployment]]"
  - "[[Sources/Homelab/ob1-deployment]]"
  - "[[Sources/Dev/email-to-obsidian-automation]]"
  - "[[Sources/Dev/email-obsidian-git-pipeline]]"
  - "[[Sources/Homelab/homelab-mcp-server]]"
  - "[[Sources/Homelab/2026-05-06-homelab-mcp-server-stand-up]]"
  - "[[Sources/Dev/wiki-vault-migration]]"
created: 2026-05-08
updated: 2026-05-08
---

# AI Memory Architecture

## Synthesis

Travis's AI memory system is not a single tool but a layered architecture: a structured wiki vault for human-readable knowledge, a pgvector semantic store for AI-queryable memory, a homelab MCP server to expose both to AI agents, and n8n pipelines to keep all layers in sync. These components appear across 8+ projects and represent a coherent long-term investment in AI-assisted knowledge work.

### The Three Memory Layers

```
Layer 1: Structured Wiki (wiki-vault)
  - Human-readable Markdown
  - Obsidian wikilinks + navigation
  - Compiled by LLM (wiki-maintenance skill)
  - Git-versioned
  - Written by: MCP server, email pipeline, Travis/Jenny

Layer 2: Semantic Memory (OB1)
  - Supabase + pgvector
  - Embeddings of vault content + session context
  - Queryable by AI agents via semantic search
  - Written by: n8n parallel pipeline (mirrors vault writes)
  - Access: Tailscale-only

Layer 3: Agent Context (Dispatch / Lovebug)
  - Active working memory within Claude Code context windows
  - Pre-compact hook (Phase 7) will drain important context → OB1
  - MCP server provides tool access to query OB1 and wiki from active sessions
```

### Ingest Paths

Multiple input channels feed into the wiki-vault and, in parallel, into OB1:

1. **MCP server** (`mcp.herbylab.dev`) — Claude Code and other AI agents write session notes and project updates directly to `Sources/` via the MCP tools. This is the primary agentic write path.

2. **Email pipeline** — Gmail → n8n → Gitea push → vault. Designed for mobile/async capture: send an email, it becomes a session note. Built and validated in the email-obsidian-git-pipeline session.

3. **Manual / direct** — Travis or Jenny write directly to the vault via Obsidian.

4. **Parallel OB1 write** (planned) — n8n intercepts all vault writes and mirrors them to OB1 as embeddings. Not yet live; pending OB1 deployment Phase 4.

### Query Paths

- **Wiki** — humans browse via Obsidian; AI agents query via MCP vault-read tools
- **OB1** — AI agents query via MCP OB1-recall tools (semantic search over embeddings)
- **Context injection** — the homelab MCP server exposes both layers as Claude Code tools, enabling Lovebug to pull relevant wiki pages or OB1 memories as needed within a session

### Harness vs. Model Architecture

The AI memory research session explored the design question: should AI memory be implemented at the **model layer** (fine-tuning, RLHF on personal data) or the **harness layer** (RAG, context injection, external retrieval)? The conclusion was harness-layer: model weights are slow and expensive to update, while a pgvector store + MCP injection is fast, reversible, and inspectable. Poolside and Pi.dev were evaluated as examples of model-layer approaches — useful for code but not for personal episodic memory.

The two memory modes are distinct:
- **Episodic** (session notes, project logs) → OB1 embeddings for temporal/event recall
- **Semantic** (wiki entity pages, synthesis) → wiki-vault for stable structured knowledge

### The Second-Brain Pattern

The `second-brain` project captures the Karpathy LLM wiki pattern: pull raw content → transcribe → extract entities/facts → review → publish to wiki. This is exactly what the wiki-maintenance compile loop implements. The compile loop is the "review → publish" step; the MCP server + email pipeline are the "pull → capture" steps. OB1 adds the AI-queryable retrieval layer that the pure wiki-markdown model lacks.

### MCP Server as the Integration Point

The homelab MCP server (`mcp.herbylab.dev`, deployed 2026-05-06) is the critical integration point that makes this architecture operational. Before MCP, each layer was separate — the wiki was browsed in Obsidian, OB1 was planned but not integrated, and AI agents had no way to query either during a session. The MCP server exposes vault tools and (once OB1 is live) OB1 recall as Claude Code tool calls, making the entire memory stack available to Lovebug mid-session.

## Sources

**ai-memory-architecture-research** (session notes) provides the harness-vs-model architecture decision, the pgvector choice, and the Poolside/Pi.dev research. This is the foundational design session for OB1.

**second-brain** (project plan) captures the Karpathy LLM wiki pattern and maps it to the wiki-vault compile loop + MCP server design. Establishes the pull→transcribe→extract→review→publish pipeline.

**ob1-main-deployment** (project plan) provides the full OB1 deployment architecture: Supabase + pgvector, Phases 0–6, embedding pipeline design.

**ob1-deployment** (addendum) captures the locked decisions: Tailscale-only access, n8n parallel write pipeline, Phase 7 Dispatch pre-compact hook. Critical refinements to the main plan.

**email-to-obsidian-automation** and **email-obsidian-git-pipeline** document the Gmail → n8n → Gitea → vault ingest path. The pipeline is built and validated as of May 2026.

**homelab-mcp-server** and **2026-05-06-homelab-mcp-server-stand-up** document the MCP server deployment: FastMCP Streamable HTTP, Authentik OAuth2, Cloudflare Tunnel. The integration point connecting all memory layers to AI agents.

**wiki-vault-migration** documents the vault consolidation from pbs-projects + homelab-projects into this unified structure, establishing the wiki-vault as the canonical structured knowledge store.

## Open questions

- OB1 Phases 0–6 deployment timeline: what is the current blocker? The addendum was written but the deployment plan doesn't indicate Phase 0 is complete.
- When the n8n parallel write pipeline goes live, will it backfill existing vault content (all 62+ notes) into OB1, or only new writes going forward?
- The Dispatch pre-compact hook (Phase 7) depends on Dispatch exposing a lifecycle event. Is this a Dispatch feature that already exists, or does it require Dispatch development work?
- How will the MCP server's OB1 recall tool handle stale embeddings when vault content is updated by the compile loop? Will there be a re-embed step in the compile procedure?
- The wiki + OB1 parallel write model creates two sources of truth for the same content. Reconciliation strategy (wiki as authoritative; OB1 as derived index) should be documented once the pipeline goes live.