wiki-vault/Sources/Dev/wiki-vault-migration.md
Travis Herbranson 34a268d8dc migration: copy 62 notes from pbs-projects and homelab-projects
Two-layer structure: Sources (raw notes) + Wiki (compile output)
Four domains: Dev (40), Venture (3), Homelab (23), Reference (0)
Includes CLAUDE.md spec, index pages at all levels, compile log

Co-Authored-By: Lovebug <lovebug@herbylab.dev>
2026-05-08 17:21:13 -04:00

263 lines
10 KiB
Markdown

---
created: 2026-05-08
path: Sources/Dev
project: wiki-vault-migration
status: active
tags:
- obsidian
- mcp
- automation
- wiki
type: project-plan
updated: 2026-05-08
---
# Wiki Vault Migration — Unified Notes Repository
## Goal
Consolidate pbs-projects and homelab-projects notes into a single
`wiki-vault` repository with a clean two-layer structure (Sources +
Wiki) under four ownership domains. Update the MCP server schema to
match. Migrate existing files without disrupting the old repos.
## Architecture
### Two-layer structure
```
wiki-vault/ (/opt/projects/wiki-vault/)
├── CLAUDE.md # vault spec (read by compile skill)
├── index.md # front door — links into Sources + Wiki
├── log.md # compile audit trail
├── Sources/ # layer 1: raw input (human + agent authored)
│ ├── index.md # sources catalog — all projects + sessions
│ ├── Dev/
│ │ ├── index.md # Dev projects + sessions listing
│ │ └── *.md # project plans, session notes — flat
│ ├── Venture/
│ │ ├── index.md
│ │ └── *.md
│ ├── Homelab/
│ │ ├── index.md
│ │ └── *.md
│ └── Reference/
│ ├── index.md
│ └── *.md
└── Wiki/ # layer 2: compiled output (agent generated)
├── index.md # wiki-wide catalog
├── Dev/
│ ├── index.md # Dev entities, topic landings, synthesis
│ └── *.md
├── Venture/
│ ├── index.md
│ └── *.md
├── Homelab/
│ ├── index.md
│ └── *.md
└── Reference/
├── index.md # cross-domain entities
└── *.md
```
### Layer responsibilities
- **Sources** — project plans and session notes. Written by Travis,
Jenny, and LLM agents via the MCP server. Immutable once written
(edits create new versions, originals stay). The MCP `save_note`
tool writes here.
- **Wiki** — entity pages, topic landings, synthesis pages. Generated
and maintained by the wiki-maintenance compile skill. Never written
to directly by the MCP server or by agents outside the compile loop.
- **Spec layer** — `CLAUDE.md`, `index.md`, `log.md` at the root.
`CLAUDE.md` is the operating spec for the compile skill. Root
`index.md` is the navigation front door.
### Navigation model
All browsing happens through index pages, not the file tree.
- **Root `index.md`** — links to Sources index, Wiki index, and
highlights (active projects, recent sessions, key entities)
- **`Sources/index.md`** — all projects grouped by status, recent
sessions by date, cross-domain view
- **`Sources/<Domain>/index.md`** — domain-specific project + session
listing
- **`Wiki/index.md`** — all entities, landings, synthesis pages
- **`Wiki/<Domain>/index.md`** — domain-specific compiled pages
The compile loop maintains all index pages. Travis navigates via
wikilinks from the front door, never by drilling into folders.
### Four ownership domains
| Domain | Scope |
|--------|-------|
| **Dev** | Coding projects, dev environment, AI/ML, DeFi, tooling, language explorations |
| **Venture** | PBS business + content: recipes, video production, brand strategy, membership, marketing, revenue, partnerships — everything Travis and Jenny are building as a business |
| **Homelab** | Infrastructure, networking, hardware, sysadmin, storage, deployments, self-hosting |
| **Reference** | Cross-domain entities: people, systems, tools that span multiple domains (Lovebug, OB1, herbydev, Jenny, Sunnie, Tailscale, etc.) |
Cross-domain content: prefer the domain where primary ownership lives.
If genuinely cross-cutting and entity-shaped, put in Reference. If
genuinely cross-cutting and project-shaped, pick one domain and tag
with the others.
## Locked Decisions
- [x] One repo (`wiki-vault`), four domains, two layers
- [x] Flat within each domain — type is in frontmatter, not folders
- [x] Domain name: Venture (not Content, PBS, Creative, or Forge)
- [x] Old repos (pbs-projects, homelab-projects) stay alive as backup
- [x] Migration copies files, does not move them
- [x] Sources indexes are compile-maintained, same as Wiki indexes
- [x] Navigation through index pages, not file tree browsing
- [x] MCP server updated: schema, folder paths, save_note targets
- [x] Wiki-maintenance skill updated: CLAUDE.md reflects new structure
- [x] Repo location: `/opt/projects/wiki-vault/`
## MCP Schema Updates
The `get_schema` tool needs these changes:
### Domain list
Replace `Content` with `Venture`. Update descriptions.
### Folder paths
```
project-plan → Sources/<Domain>/
session-notes → Sources/<Domain>/
entity → Wiki/<Domain>/ (compile-only, not MCP-writable)
topic-landing → Wiki/<Domain>/ (compile-only)
synthesis → Wiki/<Domain>/ (compile-only)
```
### Section templates (new)
Add required H2 sections per note type so agents produce consistent
body structure:
**project-plan required sections:**
- `## Goal` — one paragraph, what this project achieves
- `## Locked Decisions` — bulleted, dated as needed
- `## Open Items` — what's unresolved
- `## Phases` or `## Tasks` — work breakdown
- `## Notes` — learnings, context, references
**session-notes required sections:**
- `## Outcome` — what came out of the session
- `## Topics Covered` — what was discussed
- `## Key Learnings` — what was learned
- `## Follow-ons` — checkbox list of next steps
### Frontmatter path field
Update from `Tech/Projects``Sources/Dev`, `Sources/Venture`, etc.
## Build Phases
### Phase 1 — Create the repo and structure
- [ ] Create `/opt/projects/wiki-vault/` with full folder tree
- [ ] Initialize git repo
- [ ] Write `CLAUDE.md` (updated spec reflecting two-layer structure)
- [ ] Write stub `index.md` at root, Sources, Wiki, and all 8 domain
folders
- [ ] Write initial `log.md`
- [ ] Create Gitea repo, configure remote + deploy key
- [ ] Push initial structure
### Phase 2 — Migration script
- [ ] Write Python script that reads both source repos
- [ ] For each markdown file with frontmatter:
- Determine domain (Dev, Venture, Homelab, Reference) from existing
path and tags
- Determine type (project-plan, session-notes) from frontmatter
- Update `path:` frontmatter to new location
- Copy to `Sources/<Domain>/<filename>`
- [ ] Generate a migration report: file count per domain, any files
that couldn't be auto-categorized
- [ ] Review report with Travis before committing
- [ ] Run the copy
- [ ] Commit with message: `migration: copy N files from pbs-projects
and homelab-projects`
### Phase 3 — Update MCP server
- [ ] Update `schema.py`: domains, folder paths, section templates
- [ ] Update `git_writer.py`: vault path points to wiki-vault
- [ ] Update `save_note` in `vault.py`: writes to `Sources/<Domain>/`
- [ ] Update `list_projects` and `get_project`: scan `Sources/` tree
- [ ] Update `check_vault_file`: scan new paths
- [ ] Update `.env`: `VAULT_HOST_PATH` points to wiki-vault
- [ ] Rebuild and deploy MCP server container
- [ ] Test: save a note via claude.ai, verify it lands in the right
Sources domain folder
### Phase 4 — Update wiki-maintenance skill
- [ ] Update `CLAUDE.md` in wiki-vault with new structure spec
- [ ] Update compile procedure: reads from `Sources/`, writes to `Wiki/`
- [ ] Update lint procedure: checks both layers
- [ ] Update migration runbook (if still relevant) or mark as complete
- [ ] Reinstall skill at `~/.claude/skills/wiki-maintenance/`
### Phase 5 — Update wiki-context project
- [ ] Update wiki-context `CLAUDE.md` to reference the two-layer model
- [ ] Align domain names (Content → Venture)
- [ ] Confirm compile skill still loads and reports correctly
### Phase 6 — Initial compile run
- [ ] Trigger compile against wiki-vault
- [ ] Generate initial Sources indexes (project + session listings)
- [ ] Generate initial Wiki indexes (empty until entities are promoted)
- [ ] Generate root index.md with navigation links
- [ ] Review output, iterate on index formatting
- [ ] Open wiki-vault in Obsidian, verify navigation flow works
## File Categorization Rules (for migration script)
| Current path pattern | Target domain | Rationale |
|---------------------|---------------|-----------|
| `Tech/Projects/*` | Dev | Technical project plans |
| `Tech/Sessions/*` | Dev | Technical session notes |
| `Tech/Reference/*` | Reference | Cross-domain entities |
| `PBS/Content/*` | Venture | PBS content planning |
| `PBS/Tech/Projects/*` | Dev (tag: pbs) | PBS tech work is Dev work |
| `PBS/Tech/Sessions/*` | Dev (tag: pbs) | PBS tech sessions |
| `PBS/Inbox/*` | Dev or Venture | Manual review needed |
| `Business/*` | Venture | Business planning |
| `PBS-Planning/*` | Venture | PBS planning |
| `settings/*` | Skip | Config files, not notes |
Files without frontmatter: skip and list in the migration report.
Files with ambiguous domain: list for manual review.
## Open Items
- [ ] Decide whether `Sources/Reference/` is needed — Reference might
only contain Wiki-layer entity pages, not raw source notes. If so,
drop it from Sources.
- [ ] Confirm the n8n email pipeline can be updated to write to the
new paths (or if it stays writing to the old repos for now)
- [ ] Decide whether to update the Dispatch session plan templates to
reference wiki-vault instead of pbs-projects
## Success Criteria
- [ ] All project plans and session notes from both old repos exist in
wiki-vault under the correct domain
- [ ] MCP `save_note` writes to `Sources/<Domain>/` and commits
- [ ] MCP `get_schema` returns updated domains, paths, and section
templates
- [ ] MCP `list_projects` finds projects in the new structure
- [ ] Root `index.md` in Obsidian provides working navigation to all
content via wikilinks
- [ ] Sources and Wiki indexes are populated after first compile
- [ ] Old repos remain untouched and accessible as backup
- [ ] An agent posting a new project plan via MCP produces a file with
correct frontmatter, correct domain folder, and all required H2
sections