6722ee735b
Pages touched: Sources/index, Sources/Dev/index, Sources/Homelab/index,
Wiki/index, Wiki/Reference/index, Wiki/Reference/{ob1,traefik,gitea},
Wiki/Homelab/proxmox, Wiki/Dev/pbs-hub, Wiki/Dev/ai-memory-architecture
Pages created: none
Entities promoted: none (proposed: trellis, vault-mcp, petalbrain, pbs-hub-mcp, knot-dns, herbyeos)
Synthesis: ai-memory-architecture updated (9->17 sources) for petalbrain consolidation,
shared public.embeddings, OB1 two-layer + reflection job, JSONL watcher, Trellis
Contradictions flagged: OB1 "Supabase" -> self-hosted pgvector:pg17/petalbrain + Ollama (corrected)
Tags proposed: none
Notes: 8 pipeline test artifacts flagged for cleanup (not deleted); session-note->session-notes
lint pending; Venture unchanged
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
156 lines
12 KiB
Markdown
156 lines
12 KiB
Markdown
---
|
||
type: synthesis
|
||
path: Wiki/Dev
|
||
tags:
|
||
- ob1
|
||
- embeddings
|
||
- pgvector
|
||
- wiki
|
||
- automation
|
||
- mcp
|
||
sources:
|
||
- "[[Sources/Dev/ai-memory-architecture-research]]"
|
||
- "[[Sources/Dev/second-brain]]"
|
||
- "[[Sources/Homelab/ob1-main-deployment]]"
|
||
- "[[Sources/Homelab/ob1-deployment]]"
|
||
- "[[Sources/Dev/email-to-obsidian-automation]]"
|
||
- "[[Sources/Dev/email-obsidian-git-pipeline]]"
|
||
- "[[Sources/Homelab/homelab-mcp-server]]"
|
||
- "[[Sources/Homelab/2026-05-06-homelab-mcp-server-stand-up]]"
|
||
- "[[Sources/Dev/wiki-vault-migration]]"
|
||
- "[[Sources/Homelab/postgres-consolidation-ob1-reflection-layer]]"
|
||
- "[[Sources/Homelab/postgres-consolidation-followups]]"
|
||
- "[[Sources/Homelab/2026-05-15-postgres-consolidation-phase0-recon]]"
|
||
- "[[Sources/Homelab/2026-05-15-postgres-consolidation-phase2a-result]]"
|
||
- "[[Sources/Homelab/ob1-jsonl-watcher]]"
|
||
- "[[Sources/Homelab/vault-semantic-search]]"
|
||
- "[[Sources/Homelab/herbylab-project-management]]"
|
||
- "[[Sources/Homelab/trellis-mcp]]"
|
||
created: 2026-05-08
|
||
updated: 2026-05-24
|
||
---
|
||
|
||
# AI Memory Architecture
|
||
|
||
## Synthesis
|
||
|
||
Travis's AI memory system is not a single tool but a layered architecture: a structured wiki vault for human-readable knowledge, a pgvector semantic store for AI-queryable memory, a homelab MCP server to expose both to AI agents, and pipelines to keep all layers in sync. These components appear across 15+ projects and represent a coherent long-term investment in AI-assisted knowledge work.
|
||
|
||
> **Update (2026-05-24 compile).** Two things in the original synthesis below are now superseded and corrected in-line: (1) OB1 was never "Supabase" — it's a self-hosted `pgvector/pgvector:pg17` container, since consolidated into the `petalbrain` database; (2) the "n8n parallel write" ingest path for OB1 was replaced by a JSONL session watcher. The **Consolidated Substrate** and **Trellis: the third leg** sections below capture the May-2026 state; the older sections are kept for design-history continuity.
|
||
|
||
### The Consolidated Substrate (`petalbrain`)
|
||
|
||
As of the May-2026 postgres consolidation, all three memory systems share one Postgres instance (`homelab-postgres`, pgvector:pg17) under one database, **`petalbrain`**, with schema-level isolation and a dedicated service role per system:
|
||
|
||
```
|
||
petalbrain (Postgres / pgvector:pg17, on Proxmox, Tailscale-only)
|
||
├── schema wiki — owner vault_mcp — wiki.artifacts / projects / notes
|
||
├── schema ob1 — owner ob1_mcp — ob1.thoughts (raw) + ob1.reflections (compiled)
|
||
├── schema trellis — owner trellis_mcp — threads / events / tags
|
||
└── public.embeddings — shared vector store, all sources, 768-d nomic-embed-text
|
||
HNSW vector_cosine_ops (m=16, ef_construction=64)
|
||
512-token chunks / 64-token overlap
|
||
```
|
||
|
||
`public.embeddings` is the load-bearing idea: one table for every source, keyed by `(source_schema, source_table, source_id, embedding_model)`, so "find similar to X across everything" is a single `ORDER BY embedding <-> query` rather than a UNION across per-system tables. Cross-schema reads are granted narrowly (ob1_mcp reads wiki + trellis for reflection synthesis; lovebug is RW everywhere). Isolation is enforced by Postgres at the role level, not by trusting MCP code to behave — matching the same "separate the *who*, not the *where*" pattern used for host users and Authentik.
|
||
|
||
The embedding model is **local Ollama `nomic-embed-text` (768-d)** — never a hosted provider for memory. Switching it is a deliberate full re-embed, which is why `embedding_model` is part of the uniqueness key.
|
||
|
||
### OB1's own two layers
|
||
|
||
OB1 now mirrors the wiki's Sources/Wiki split internally: `ob1.thoughts` is the raw capture layer; `ob1.reflections` is a compiled layer written by a **daily reflection job** (`claude-sonnet-4-6` via the Anthropic Messages API direct) that synthesizes generalizable lessons with citations back to source rows. Same discipline as the wiki — mixing raw and synthesized content in one table pollutes semantic neighbors. Ambient capture is the [[Sources/Homelab/ob1-jsonl-watcher|JSONL watcher]]: it tails Claude Code / Cowork session files and posts turn-pairs to OB1's `capture_thought`. "Save everything, filter later."
|
||
|
||
### Trellis: the third leg
|
||
|
||
The original two-system framing (wiki + OB1) is now three: [[Sources/Homelab/trellis-mcp|Trellis]] is resumable *working state* — named threads with checkpoints that survive conversation boundaries. The clean separation the stack settled on: **CLAUDE.md = instructions (how to behave), Trellis = record (where we are in the doing), wiki + OB1 = lessons and outcomes (what we learned/built).** None should swallow another's job. Trellis lives in `petalbrain.trellis`; its tool surface is being refined ([[Sources/Homelab/trellis-mcp-tool-surface-followups|checkpoint_thread rename + id addressing]]).
|
||
|
||
### The Three Memory Layers
|
||
|
||
```
|
||
Layer 1: Structured Wiki (wiki-vault)
|
||
- Human-readable Markdown
|
||
- Obsidian wikilinks + navigation
|
||
- Compiled by LLM (wiki-maintenance skill)
|
||
- Git-versioned
|
||
- Written by: MCP server, email pipeline, Travis/Jenny
|
||
|
||
Layer 2: Semantic Memory (OB1) [see "Consolidated Substrate" above for current state]
|
||
- self-hosted pgvector:pg17 (petalbrain), NOT Supabase
|
||
- Embeddings of vault content + session context (Ollama nomic-embed-text, 768-d)
|
||
- Queryable by AI agents via semantic search over public.embeddings
|
||
- Written by: JSONL session watcher → capture_thought (NOT n8n parallel write)
|
||
- Access: Tailscale-only
|
||
|
||
Layer 3: Agent Context (Dispatch / Lovebug)
|
||
- Active working memory within Claude Code context windows
|
||
- Pre-compact hook (Phase 7) will drain important context → OB1
|
||
- MCP server provides tool access to query OB1 and wiki from active sessions
|
||
```
|
||
|
||
### Ingest Paths
|
||
|
||
Multiple input channels feed into the wiki-vault and, in parallel, into OB1:
|
||
|
||
1. **MCP server** (`mcp.herbylab.dev`) — Claude Code and other AI agents write session notes and project updates directly to `Sources/` via the MCP tools. This is the primary agentic write path.
|
||
|
||
2. **Email pipeline** — Gmail → n8n → Gitea push → vault. Designed for mobile/async capture: send an email, it becomes a session note. Built and validated in the email-obsidian-git-pipeline session.
|
||
|
||
3. **Manual / direct** — Travis or Jenny write directly to the vault via Obsidian.
|
||
|
||
4. **Parallel OB1 write** (planned) — n8n intercepts all vault writes and mirrors them to OB1 as embeddings. Not yet live; pending OB1 deployment Phase 4.
|
||
|
||
### Query Paths
|
||
|
||
- **Wiki** — humans browse via Obsidian; AI agents query via MCP vault-read tools
|
||
- **OB1** — AI agents query via MCP OB1-recall tools (semantic search over embeddings)
|
||
- **Context injection** — the homelab MCP server exposes both layers as Claude Code tools, enabling Lovebug to pull relevant wiki pages or OB1 memories as needed within a session
|
||
|
||
### Harness vs. Model Architecture
|
||
|
||
The AI memory research session explored the design question: should AI memory be implemented at the **model layer** (fine-tuning, RLHF on personal data) or the **harness layer** (RAG, context injection, external retrieval)? The conclusion was harness-layer: model weights are slow and expensive to update, while a pgvector store + MCP injection is fast, reversible, and inspectable. Poolside and Pi.dev were evaluated as examples of model-layer approaches — useful for code but not for personal episodic memory.
|
||
|
||
The two memory modes are distinct:
|
||
- **Episodic** (session notes, project logs) → OB1 embeddings for temporal/event recall
|
||
- **Semantic** (wiki entity pages, synthesis) → wiki-vault for stable structured knowledge
|
||
|
||
### The Second-Brain Pattern
|
||
|
||
The `second-brain` project captures the Karpathy LLM wiki pattern: pull raw content → transcribe → extract entities/facts → review → publish to wiki. This is exactly what the wiki-maintenance compile loop implements. The compile loop is the "review → publish" step; the MCP server + email pipeline are the "pull → capture" steps. OB1 adds the AI-queryable retrieval layer that the pure wiki-markdown model lacks.
|
||
|
||
### MCP Server as the Integration Point
|
||
|
||
The homelab MCP server (`mcp.herbylab.dev`, deployed 2026-05-06) is the critical integration point that makes this architecture operational. Before MCP, each layer was separate — the wiki was browsed in Obsidian, OB1 was planned but not integrated, and AI agents had no way to query either during a session. The MCP server exposes vault tools and (once OB1 is live) OB1 recall as Claude Code tool calls, making the entire memory stack available to Lovebug mid-session.
|
||
|
||
## Sources
|
||
|
||
**ai-memory-architecture-research** (session notes) provides the harness-vs-model architecture decision, the pgvector choice, and the Poolside/Pi.dev research. This is the foundational design session for OB1.
|
||
|
||
**second-brain** (project plan) captures the Karpathy LLM wiki pattern and maps it to the wiki-vault compile loop + MCP server design. Establishes the pull→transcribe→extract→review→publish pipeline.
|
||
|
||
**ob1-main-deployment** (project plan) provides the full OB1 deployment architecture: Supabase + pgvector, Phases 0–6, embedding pipeline design.
|
||
|
||
**ob1-deployment** (addendum) captures the locked decisions: Tailscale-only access, n8n parallel write pipeline, Phase 7 Dispatch pre-compact hook. Critical refinements to the main plan.
|
||
|
||
**email-to-obsidian-automation** and **email-obsidian-git-pipeline** document the Gmail → n8n → Gitea → vault ingest path. The pipeline is built and validated as of May 2026.
|
||
|
||
**homelab-mcp-server** and **2026-05-06-homelab-mcp-server-stand-up** document the MCP server deployment: FastMCP Streamable HTTP, Authentik OAuth2, Cloudflare Tunnel. The integration point connecting all memory layers to AI agents.
|
||
|
||
**wiki-vault-migration** documents the vault consolidation from pbs-projects + homelab-projects into this unified structure, establishing the wiki-vault as the canonical structured knowledge store.
|
||
|
||
**postgres-consolidation-ob1-reflection-layer** (project plan) is the source for the single-database / three-schema / four-service-user model, the shared `public.embeddings` design, and OB1's two-layer captures+reflections split. **2026-05-15-postgres-consolidation-phase0-recon** is the primary evidence that OB1 is pgvector:pg17 + Ollama `nomic-embed-text` (768-d), not Supabase — it inventories the live instance directly. **phase2a-result** confirms the schemas, roles, and `public.embeddings` table landed with verified permission isolation. **postgres-consolidation-followups** supplies the reflection-model choice (`claude-sonnet-4-6` direct), chunking parameters (512/64), and the enricher loop bug.
|
||
|
||
**ob1-jsonl-watcher** (project plan) is the source for the ambient-capture-via-JSONL design that replaced the n8n parallel-write idea. **trellis-mcp** establishes the third memory leg and the instructions/record/lessons three-way separation. **herbylab-project-management** documents the Postgres-backed artifact model and the NOTIFY→n8n→Trello projection. **vault-semantic-search** specifies the (still-unbuilt) `vault:search` tool over the wiki schema.
|
||
|
||
## Open questions
|
||
|
||
*Several earlier open questions were resolved by the May-2026 consolidation: OB1's substrate is confirmed (pgvector:pg17 / `petalbrain`, not Supabase); ambient capture is the JSONL watcher, not n8n parallel-write (so there is no wiki↔OB1 dual-source-of-truth problem — vault is content, OB1 indexes session content); and the shared `public.embeddings` table is live.*
|
||
|
||
Still open:
|
||
|
||
- **Reflector quality** — the daily reflection job's synthesis prompt is unproven; first non-dry-run output needs review before the 3am timer is trusted (postgres-consolidation-followups Phase 1).
|
||
- **`ob1-enricher` PENDING_QUERY loop** — must be fixed before the enricher is restarted, or the metadata backlog grows.
|
||
- **Re-embed on compile** — when the wiki compile loop edits a Wiki page, its `public.embeddings` row goes stale. There is still no re-embed step in the compile procedure. *(Agent inference, not stated in any source — flagged.)*
|
||
- **`vault:search` not yet built** — the semantic-search MCP tool over the wiki schema is designed ([[Sources/Homelab/vault-semantic-search|vault-semantic-search]]) but unbuilt; until it ships, the wiki half of the substrate isn't actually queryable by agents.
|
||
- **claude/competent-knuth-de9483 branch** — artifact tooling that landed via a separate path; merge/abandon/cherry-pick still undecided.
|
||
- **Dispatch pre-compact hook (Phase 7)** — still depends on a Dispatch lifecycle event; the JSONL watcher covers Claude Code/Cowork capture in the meantime.
|