63 lines
2.0 KiB
Markdown
63 lines
2.0 KiB
Markdown
# second-brain
|
|
|
|
A personal knowledge system: video/article extraction pipeline + LLM-maintained wiki.
|
|
|
|
## Architecture
|
|
|
|
Three-layer system:
|
|
|
|
1. **Wiki** — long-term structured memory. Markdown files in an Obsidian vault, maintained by the wiki compiler.
|
|
2. **Context assembler** — builds the prompt bundle for each extraction. Phase 1 is thin: domain template + focus field + transcript.
|
|
3. **Extractor** — stateless single-shot LLM call per source. Pure function: bundle in, structured extraction out.
|
|
|
|
## Pipeline stages
|
|
|
|
```
|
|
pending → pulled → transcribed → analyzed → accepted → published
|
|
```
|
|
|
|
## Project layout
|
|
|
|
```
|
|
src/second_brain/
|
|
├── adapters/ # Pull adapters (youtube.py, article.py)
|
|
├── context/ # Context assembler
|
|
├── extractor/ # LLM extraction engine + Pydantic schema
|
|
├── compiler/ # Wiki compiler (reads vault, writes pages, git commits)
|
|
├── scheduler/ # Rate-limit smoother, token-aware spacing
|
|
└── web/ # FastAPI + Jinja2 + HTMX UI
|
|
```
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
cp config/settings.toml.example config/settings.toml
|
|
# Edit settings.toml with your paths
|
|
uv run second-brain serve
|
|
```
|
|
|
|
## CLI commands
|
|
|
|
- `second-brain add <url>` — queue a source URL (video or article)
|
|
- `second-brain process` — run the pipeline on pending items
|
|
- `second-brain serve` — launch the web UI (default port 8000)
|
|
- `second-brain compile` — run the wiki compiler
|
|
- `second-brain schedule` — start the overnight scheduler
|
|
|
|
## Environment variables
|
|
|
|
- `ANTHROPIC_API_KEY` — required for extraction and wiki compilation
|
|
|
|
## Domains
|
|
|
|
- `development` — software, programming, systems
|
|
- `content` — content creation strategy, video production
|
|
- `business` — entrepreneurship, marketing, operations
|
|
- `homelab` — self-hosted infrastructure, networking, DevOps
|
|
|
|
## Key files
|
|
|
|
- `config/settings.toml` — vault path, DB path, scheduler window
|
|
- `prompts/<domain>.md` — per-domain extraction prompt templates
|
|
- `src/second_brain/extractor/schema.py` — canonical extraction output schema
|