214 lines
7.5 KiB
Markdown
214 lines
7.5 KiB
Markdown
---
|
||
project: second-brain
|
||
type: project-plan
|
||
status: active
|
||
path: Tech/Projects
|
||
tags:
|
||
- homelab
|
||
- llm
|
||
- obsidian
|
||
- knowledge-management
|
||
- claude-code
|
||
created: 2026-05-04
|
||
updated: 2026-05-04
|
||
---
|
||
# Second Brain — Project Plan
|
||
|
||
## Concept
|
||
|
||
A personal knowledge system combining a video/article extraction pipeline
|
||
with an LLM-maintained wiki (Karpathy LLM Wiki pattern). Sources get
|
||
pulled, transcribed/extracted, reviewed, and compiled into an
|
||
Obsidian-resident wiki that grows and self-maintains over time.
|
||
|
||
**Phase 1 success:** working knowledge extractor + working LLM wiki,
|
||
end-to-end on one vertical slice, generalizable to all four domains.
|
||
|
||
**Phase 2+ (out of scope for now):** the "second brain" layer — static
|
||
user-context, active concerns, recent decisions, open questions, wiki-aware
|
||
extraction. Stretch goal, fun experiment, not a Phase 1 gate.
|
||
|
||
## Architecture (3 layers)
|
||
|
||
1. **Wiki** — long-term structured memory. Markdown files in Obsidian
|
||
vault, LLM-maintained. Karpathy three-layer pattern: raw sources
|
||
(immutable) → wiki (LLM-generated) → schema/config.
|
||
2. **Context assembler** — module that builds the prompt bundle for each
|
||
extraction. Phase 1 is thin (domain template + focus field + transcript)
|
||
but built as a real module from day one so Phase 2 layers slot in cleanly.
|
||
3. **Extractor** — stateless single-shot LLM call per source. Pure
|
||
function: bundle in, structured extraction out.
|
||
|
||
## Pipeline
|
||
|
||
|
||
pull → transcribe → extract → review → publish (LLM-compiled to wiki)
|
||
|
||
|
||
Existing tool already has pull/transcribe/extract and a Flask web UI with
|
||
list/detail/accept. Extending it, not rewriting.
|
||
|
||
## Decisions locked
|
||
|
||
### Scope
|
||
- [x] Phase 1: extractor + wiki, all manual review
|
||
- [x] Phase 2 deferred: user-context, dynamic concerns, wiki-aware
|
||
extraction
|
||
|
||
### Sources & domains
|
||
- [x] Sources: web pages, videos
|
||
- [x] Domains: `development`, `content`, `business`, `homelab`
|
||
|
||
### Scheduler
|
||
- [x] Rate-limit smoother, not a batch processor
|
||
- [x] Spaces N items across an overnight window (e.g., 10pm–6am)
|
||
- [x] Token-aware budgeting preferred over pure time spacing (track tokens
|
||
per job)
|
||
- [x] Failures retry with backoff, dead-letter after N attempts
|
||
- [x] Idempotent — re-running a finished URL is a no-op
|
||
|
||
### LLM access
|
||
- [x] Claude Code Python SDK (subscription-based, not API)
|
||
- [x] Single-shot calls — no agent loop, no tool use, no session continuity
|
||
- [x] `--bare` mode equivalent for reproducibility
|
||
- [x] Subscription auth on herbydev — needs verification before scheduler
|
||
goes live
|
||
|
||
### Data model
|
||
- [x] Per-job `domain` field (one of four)
|
||
- [x] Per-job `focus` field (optional, free text — evaluative framing hint)
|
||
- [x] Status states: `pending → pulled → transcribed → analyzed → accepted
|
||
→ published`
|
||
- [x] Clean slate — no legacy data to migrate
|
||
|
||
### Extraction output schema (contract between extractor and compiler)
|
||
yaml
|
||
source:
|
||
url, title, type, duration_or_length, published_at, ingested_at
|
||
domain:
|
||
focus:
|
||
summary:
|
||
key_points: [...]
|
||
entities: [...] # candidate wiki entities
|
||
claims: [...] # discrete claims tied to key_points
|
||
open_questions: [...]
|
||
contradictions: [...]
|
||
|
||
|
||
### Wiki publish step
|
||
- [x] Option 3: LLM-compiled (full Karpathy pattern, not templated, not
|
||
manual)
|
||
- [x] Compiler reads existing wiki, decides what to update/create, writes
|
||
changes
|
||
- [x] Auto-accept on compiler writes (no diff/dry-run gate in Phase 1)
|
||
- [x] Git-as-recovery: commit before each compile run, commit after
|
||
- Pre-write commit: `pre-compile: `
|
||
- Post-write commit: `compile: ` with reasoning in body
|
||
(entities touched, created, flagged)
|
||
- `git log` becomes the audit trail; `git reset` is the rollback
|
||
|
||
### Wiki page types
|
||
- [x] Default to LLM Wiki spec — include all types (entity, concept,
|
||
comparison, synthesis)
|
||
- [x] Skinny down later after running it
|
||
|
||
### Source provenance
|
||
- [x] Default to LLM Wiki spec
|
||
|
||
### Compiler decision authority
|
||
- [x] Default to LLM Wiki spec
|
||
|
||
## Open items
|
||
|
||
### Wiki folder structure
|
||
- [ ] Review LLM Wiki reference projects before deciding flat vs. nested
|
||
- [ ] Lean: may flatten (lots of folders not helpful)
|
||
- [ ] Decision needed before compiler is wired to vault
|
||
|
||
### Reference implementations to review
|
||
- [ ] `Ar9av/obsidian-wiki` — skill-files for any agent, points at Obsidian
|
||
vault, provenance tagging
|
||
- [ ] `lucasastorian/llmwiki` — local app + MCP, indexes folder, Claude
|
||
writes pages
|
||
- [ ] `nvk/llm-wiki` — Claude Code plugin, parallel multi-agent research
|
||
- [ ] Karpathy original gist — abstract spec
|
||
|
||
### Verification before scheduler runs
|
||
- [ ] Subscription auth in non-interactive cron context on herbydev
|
||
|
||
## Build approach
|
||
|
||
**Vertical slice first.** Pick one domain (likely `homelab`) + one source
|
||
type (likely YouTube). Build pull → transcribe → extract → review → publish
|
||
end-to-end. Generalizing to other domains is mostly config; adding article
|
||
ingestion is one new adapter.
|
||
|
||
## Handoff to dev
|
||
|
||
This plan covers the *what* and *why*. Dev session on herbydev handles the
|
||
*where* and *how-to-wire-it-up*:
|
||
- Repo layout under `~/dev/`
|
||
- Integration with existing extraction tool (lift working pieces, fresh
|
||
codebase)
|
||
- Obsidian vault path on herbydev
|
||
- Scheduler host/process structure (LXC, systemd timer, etc.)
|
||
- Existing extractor code review and selective port
|
||
|
||
## Phase 1 explicitly out of scope
|
||
- Static user-context file
|
||
- Active concerns / recent decisions / open questions context layers
|
||
- Wiki-aware extraction (extractor doesn't read existing pages)
|
||
- Auto-accept on extractions (always lands in `analyzed` for review)
|
||
- LLM-maintained context
|
||
- Diff/dry-run mode on compiler writes
|
||
|
||
## Notes & learning
|
||
|
||
- LLM Wiki pattern (Karpathy, April 2026): three layers — immutable
|
||
sources, LLM-maintained wiki, schema/config file. Knowledge compiled once,
|
||
kept current — not re-derived per query like RAG.
|
||
- The "feeling of memory" in LLM systems is harness-driven, not
|
||
model-driven. Model is stateless; harness shapes the bundle. Phase 2
|
||
"second brain" work is fundamentally context-assembler design, not
|
||
extractor design.
|
||
- SDK exercise: Claude Code Python SDK chosen partly for learning value.
|
||
Uses ~10% of its surface for Phase 1 (single-shot calls), but that's fine —
|
||
it's the right glue.
|
||
- Real intellectual work in Phase 2 lives in the context assembler, not the
|
||
extractor or the wiki. That's where the second-brain feeling actually comes
|
||
from.
|
||
|
||
## Tasks
|
||
|
||
### Pre-build
|
||
- [ ] Review LLM Wiki reference repos (30 min)
|
||
- [ ] Decide wiki folder structure (flat vs. nested)
|
||
- [ ] Verify subscription auth in cron on herbydev
|
||
|
||
### Build (vertical slice)
|
||
- [ ] Repo layout + project skeleton (dev session)
|
||
- [ ] Port working pieces from existing extractor (dev session)
|
||
- [ ] Context assembler module (thin, but real)
|
||
- [ ] Extractor with Claude Code Python SDK
|
||
- [ ] Per-domain prompt template files
|
||
- [ ] Extraction output schema validation
|
||
- [ ] Queue/scheduler with token-aware spacing
|
||
- [ ] Web UI: domain filter, status filter, queue view, accept/publish
|
||
actions
|
||
- [ ] Wiki compiler module (reads vault, writes vault, two commits per run)
|
||
- [ ] End-to-end test on one homelab YouTube video
|
||
|
||
### Generalize
|
||
- [ ] Add remaining three domains
|
||
- [ ] Add web article adapter
|
||
- [ ] Run for a week, observe, tune prompts
|
||
|
||
### Phase 1 done when
|
||
- [ ] All four domains have working prompt templates
|
||
- [ ] Both source types ingest cleanly
|
||
- [ ] Scheduler runs unattended overnight without intervention
|
||
- [ ] Wiki has 20+ entity pages from real sources, with cross-references
|
||
- [ ] Git history shows clean compile commits with usable audit trail
|
||
|
||
|
||
...sent from Jenny & Travis |