---
project: second-brain
type: project-plan
status: active
path: Tech/Projects
tags:
  - homelab
  - llm
  - obsidian
  - knowledge-management
  - claude-code
created: 2026-05-04
updated: 2026-05-04
---
# Second Brain — Project Plan

## Concept

A personal knowledge system combining a video/article extraction pipeline
with an LLM-maintained wiki (Karpathy LLM Wiki pattern). Sources get
pulled, transcribed/extracted, reviewed, and compiled into an
Obsidian-resident wiki that grows and self-maintains over time.

**Phase 1 success:** working knowledge extractor + working LLM wiki,
end-to-end on one vertical slice, generalizable to all four domains.

**Phase 2+ (out of scope for now):** the "second brain" layer — static
user-context, active concerns, recent decisions, open questions, wiki-aware
extraction. Stretch goal, fun experiment, not a Phase 1 gate.

## Architecture (3 layers)

1. **Wiki** — long-term structured memory. Markdown files in Obsidian
vault, LLM-maintained. Karpathy three-layer pattern: raw sources
(immutable) → wiki (LLM-generated) → schema/config.
2. **Context assembler** — module that builds the prompt bundle for each
extraction. Phase 1 is thin (domain template + focus field + transcript)
but built as a real module from day one so Phase 2 layers slot in cleanly.
3. **Extractor** — stateless single-shot LLM call per source. Pure
function: bundle in, structured extraction out.

## Pipeline


pull → transcribe → extract → review → publish (LLM-compiled to wiki)


Existing tool already has pull/transcribe/extract and a Flask web UI with
list/detail/accept. Extending it, not rewriting.

## Decisions locked

### Scope
- [x] Phase 1: extractor + wiki, all manual review
- [x] Phase 2 deferred: user-context, dynamic concerns, wiki-aware
extraction

### Sources & domains
- [x] Sources: web pages, videos
- [x] Domains: `development`, `content`, `business`, `homelab`

### Scheduler
- [x] Rate-limit smoother, not a batch processor
- [x] Spaces N items across an overnight window (e.g., 10pm–6am)
- [x] Token-aware budgeting preferred over pure time spacing (track tokens
per job)
- [x] Failures retry with backoff, dead-letter after N attempts
- [x] Idempotent — re-running a finished URL is a no-op

### LLM access
- [x] Claude Code Python SDK (subscription-based, not API)
- [x] Single-shot calls — no agent loop, no tool use, no session continuity
- [x] `--bare` mode equivalent for reproducibility
- [x] Subscription auth on herbydev — needs verification before scheduler
goes live

### Data model
- [x] Per-job `domain` field (one of four)
- [x] Per-job `focus` field (optional, free text — evaluative framing hint)
- [x] Status states: `pending → pulled → transcribed → analyzed → accepted
→ published`
- [x] Clean slate — no legacy data to migrate

### Extraction output schema (contract between extractor and compiler)
yaml
source:
  url, title, type, duration_or_length, published_at, ingested_at
domain: 
focus: 
summary: 
key_points: [...]
entities: [...]      # candidate wiki entities
claims: [...]        # discrete claims tied to key_points
open_questions: [...]
contradictions: [...]


### Wiki publish step
- [x] Option 3: LLM-compiled (full Karpathy pattern, not templated, not
manual)
- [x] Compiler reads existing wiki, decides what to update/create, writes
changes
- [x] Auto-accept on compiler writes (no diff/dry-run gate in Phase 1)
- [x] Git-as-recovery: commit before each compile run, commit after
  - Pre-write commit: `pre-compile: `
  - Post-write commit: `compile: ` with reasoning in body
(entities touched, created, flagged)
  - `git log` becomes the audit trail; `git reset` is the rollback

### Wiki page types
- [x] Default to LLM Wiki spec — include all types (entity, concept,
comparison, synthesis)
- [x] Skinny down later after running it

### Source provenance
- [x] Default to LLM Wiki spec

### Compiler decision authority
- [x] Default to LLM Wiki spec

## Open items

### Wiki folder structure
- [ ] Review LLM Wiki reference projects before deciding flat vs. nested
- [ ] Lean: may flatten (lots of folders not helpful)
- [ ] Decision needed before compiler is wired to vault

### Reference implementations to review
- [ ] `Ar9av/obsidian-wiki` — skill-files for any agent, points at Obsidian
vault, provenance tagging
- [ ] `lucasastorian/llmwiki` — local app + MCP, indexes folder, Claude
writes pages
- [ ] `nvk/llm-wiki` — Claude Code plugin, parallel multi-agent research
- [ ] Karpathy original gist — abstract spec

### Verification before scheduler runs
- [ ] Subscription auth in non-interactive cron context on herbydev

## Build approach

**Vertical slice first.** Pick one domain (likely `homelab`) + one source
type (likely YouTube). Build pull → transcribe → extract → review → publish
end-to-end. Generalizing to other domains is mostly config; adding article
ingestion is one new adapter.

## Handoff to dev

This plan covers the *what* and *why*. Dev session on herbydev handles the
*where* and *how-to-wire-it-up*:
- Repo layout under `~/dev/`
- Integration with existing extraction tool (lift working pieces, fresh
codebase)
- Obsidian vault path on herbydev
- Scheduler host/process structure (LXC, systemd timer, etc.)
- Existing extractor code review and selective port

## Phase 1 explicitly out of scope
- Static user-context file
- Active concerns / recent decisions / open questions context layers
- Wiki-aware extraction (extractor doesn't read existing pages)
- Auto-accept on extractions (always lands in `analyzed` for review)
- LLM-maintained context
- Diff/dry-run mode on compiler writes

## Notes & learning

- LLM Wiki pattern (Karpathy, April 2026): three layers — immutable
sources, LLM-maintained wiki, schema/config file. Knowledge compiled once,
kept current — not re-derived per query like RAG.
- The "feeling of memory" in LLM systems is harness-driven, not
model-driven. Model is stateless; harness shapes the bundle. Phase 2
"second brain" work is fundamentally context-assembler design, not
extractor design.
- SDK exercise: Claude Code Python SDK chosen partly for learning value.
Uses ~10% of its surface for Phase 1 (single-shot calls), but that's fine —
it's the right glue.
- Real intellectual work in Phase 2 lives in the context assembler, not the
extractor or the wiki. That's where the second-brain feeling actually comes
from.

## Tasks

### Pre-build
- [ ] Review LLM Wiki reference repos (30 min)
- [ ] Decide wiki folder structure (flat vs. nested)
- [ ] Verify subscription auth in cron on herbydev

### Build (vertical slice)
- [ ] Repo layout + project skeleton (dev session)
- [ ] Port working pieces from existing extractor (dev session)
- [ ] Context assembler module (thin, but real)
- [ ] Extractor with Claude Code Python SDK
- [ ] Per-domain prompt template files
- [ ] Extraction output schema validation
- [ ] Queue/scheduler with token-aware spacing
- [ ] Web UI: domain filter, status filter, queue view, accept/publish
actions
- [ ] Wiki compiler module (reads vault, writes vault, two commits per run)
- [ ] End-to-end test on one homelab YouTube video

### Generalize
- [ ] Add remaining three domains
- [ ] Add web article adapter
- [ ] Run for a week, observe, tune prompts

### Phase 1 done when
- [ ] All four domains have working prompt templates
- [ ] Both source types ingest cleanly
- [ ] Scheduler runs unattended overnight without intervention
- [ ] Wiki has 20+ entity pages from real sources, with cross-references
- [ ] Git history shows clean compile commits with usable audit trail


...sent from Jenny & Travis