petal-power/second-brain
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
repo for storing the second brain project
23 Commits 4 Branches 0 Tags 1.1 MiB
Python 82.4%
HTML 15.5%
Shell 1%
Dockerfile 0.9%
Mako 0.2%
2b40157a0d
Go to file
Travis Herbranson 2b40157a0d web: add-to-queue form on /queue too, with per-page HTMX dispatch 
Travis's original ask put the form on the "queue page" but it landed
only on /dashboard. Putting it on /queue too without duplicating any
logic:

- _add_to_queue_form.html — extracted the form markup into a shared
  partial. Callers set `{% set swap_target = "#<id>-body" %}` before
  including it; that's the only knob that differs between the two
  pages. Same POST endpoint, same field set, same flash behaviour.

- _dashboard_body.html — now {% include %}s the shared partial with
  swap_target="#dashboard-body". Net change: a six-line include
  replacing the inline form.

- _queue_body.html (new) — `<div id="queue-body">` wrapping the shared
  form (swap_target="#queue-body") + the in-flight source list (same
  card markup as the Sources view, with the wrap-fix already applied).
  Empty state copy matches the queue context.

- queue.html (new) — extends base.html, page-titles "Queue", and
  includes _queue_body.html. /queue no longer reuses index.html.

- /queue route — renders queue.html via a new _build_queue_context
  helper. Same in-flight filter as before (PENDING/PULLED/TRANSCRIBED).

- POST /sources/add — reads HX-Current-URL (HTMX sends it on every
  request; falls back to Referer) and picks the response partial:
  /queue → _queue_body.html, anything else → _dashboard_body.html. One
  endpoint, one service call, two render branches — neither side
  reaches into the other's state.

Live-verified through the rebuilt container:
- GET /queue (LAN + brain.herbylab.dev) — form present.
- GET /dashboard — form still present (unchanged behaviour).
- GET / — form still absent (Sources view stays clean).
- POST from /queue → response carries id="queue-body" (and not
  dashboard-body); the freshly-added row appears in the swapped list.
- POST from /dashboard → response carries id="dashboard-body".
- Flash scenarios from /queue: queued / duplicate / playlist 13 videos
  / validation error — all four render correctly.
2026-05-25 14:21:02 -04:00
alembic transcripts: capture segment-level output into a new JSONB column 2026-05-25 13:40:49 -04:00
config postgres migration: schema, models, embeddings, alembic 2026-05-24 22:46:48 -04:00
deploy deploy/web: containerized FastAPI UI on the homelab docker network 2026-05-25 11:48:37 -04:00
prompts project init 2026-05-22 19:08:22 -04:00
reviews docs: update CLAUDE.md + migration plan to reflect shipped Postgres design 2026-05-24 22:57:31 -04:00
src/second_brain web: add-to-queue form on /queue too, with per-page HTMX dispatch 2026-05-25 14:21:02 -04:00
tests playlists: queue-time YouTube fan-out via yt-dlp extract_flat 2026-05-25 13:59:35 -04:00
.gitignore add .gitignore 2026-05-24 21:10:53 -04:00
alembic.ini postgres migration: schema, models, embeddings, alembic 2026-05-24 22:46:48 -04:00
CLAUDE.md docs: update CLAUDE.md + migration plan to reflect shipped Postgres design 2026-05-24 22:57:31 -04:00
postgres-migration-planning.md docs: update CLAUDE.md + migration plan to reflect shipped Postgres design 2026-05-24 22:57:31 -04:00
pyproject.toml pipeline split: tower transcribe worker + claim queue + faster-whisper 2026-05-25 10:11:37 -04:00
README.md updates to project files 2026-05-24 22:23:59 -04:00
uv.lock pipeline split: tower transcribe worker + claim queue + faster-whisper 2026-05-25 10:11:37 -04:00

README.md

second-brain

A personal knowledge system built on a video/article extraction pipeline and an LLM-maintained wiki (inspired by the Karpathy LLM Wiki pattern).

What it does

  1. Pull — download YouTube videos or fetch web articles
  2. Transcribe — extract transcripts via Whisper (videos) or trafilatura (articles)
  3. Extract — single-shot Claude call produces structured notes per source
  4. Review — web UI to accept or reject extractions
  5. Compile — wiki compiler folds accepted extractions into a living Obsidian vault

Quick start

# Install dependencies
uv sync

# Copy and edit config
cp config/settings.toml config/settings.local.toml

# Queue a source
uv run second-brain add https://www.youtube.com/watch?v=...

# Run the pipeline
uv run second-brain process

# Review in the web UI
uv run second-brain serve

# Compile to wiki
uv run second-brain compile

Domains

Extractions are tagged by domain so the right prompt template is used:

Domain Focus
development Software, programming, systems
content Content creation, video production
business Entrepreneurship, marketing, ops
homelab Self-hosted infra, networking, DevOps

Extractor backends

Two backends, selectable via [extractor].backend in config/settings.toml:

  • cli (default) — shells out to claude -p --output-format json --json-schema .... Runs under your Max OAuth subscription, no per-token billing. The wrapper spawns the CLI in a throwaway tempfile.TemporaryDirectory and strips ANTHROPIC_API_KEY / ANTHROPIC_AUTH_TOKEN from the env so host CLAUDE.md, hooks, and settings don't leak in, and the subscription is used instead of the API key.
  • api — uses the anthropic Python SDK, requires ANTHROPIC_API_KEY. Install with uv sync --extra api (the SDK is an optional dependency).

Requirements

  • Python 3.12+
  • Either the claude CLI on PATH (default backend) or ANTHROPIC_API_KEY (api backend)
  • ffmpeg (for Whisper audio extraction)

In-flight work

  • Postgres + pgvector migration — SQLite is the current backing store but the project is moving to Travis's Postgres instance. Planning questions live in postgres-migration-planning.md; nothing has been migrated yet.