Adds per-record edit mode and retry to the source detail page, plus a
CLI `second-brain retry <id>` for parity.
Edit mode exposes domain (select), title, focus on /sources/{id}.
URL stays read-only — it's the UNIQUE dedupe key and the embeddings
identity tuple. Save goes through a new service helper
`update_source_metadata` so the web route stays a thin wrapper.
Retry exposes two modes, both available on any record:
- full: status -> PENDING (re-pull + re-process)
- extract: status -> TRANSCRIBED (re-extract only, requires an
existing transcript; offered when source_type=video and a
transcript is present, to avoid the expensive re-download path)
Both modes clear claimed_by/claimed_at and error_message so the queue
dance picks it up cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pasting a YouTube playlist URL into either entry point now expands
into one source row per video. Single-video URLs and non-YouTube URLs
keep their existing behaviour untouched.
Service module additions:
- is_youtube_playlist_url(url): strict detector. Only `/playlist?list=…`
on a known YouTube host (youtube.com / m / music / no-www) counts.
A `watch?v=…&list=…` URL is ambiguous (user usually pasted a single
video that happens to sit inside a playlist) and intentionally falls
through to single-add. To fan out, paste the canonical playlist URL.
- expand_youtube_playlist(url, *, max_items=50): yt-dlp with
extract_flat=True, playlistend=max_items, skip_download. Builds a
canonical https://www.youtube.com/watch?v={id} URL per entry and
silently drops placeholders for private/removed videos.
- add_playlist(sess, *, url, domain, focus, max_items, expander=None):
loops expansion entries through add_source so URL validation,
source_type detection, and the UNIQUE dedupe path stay identical to
the single-add flow. Per-entry titles win over any caller-supplied
title (a single playlist title would be wrong for N videos). Partial
failures don't abort the batch — failed entries are tallied with
up-to-10 (url, reason) tuples for the flash. `expander` is an
injection seam for tests so the suite never hits YouTube unless
explicitly opted in.
- DEFAULT_PLAYLIST_MAX_ITEMS = 50 — shared ceiling, no throttle change.
CLI: `second-brain add <playlist-url>` auto-detects and reports
`expanded / added / duplicates / failed`. No new flag needed.
Web: POST /sources/add same detection. _playlist_flash() builds the
HTMX flash — "Queued N videos (M duplicates skipped, F failed)"
with sensible plural forms and graceful omission of zero counters.
Tests:
- 15 pure-Python detection cases (positives + negatives, including the
ambiguous watch?v=…&list=… rule).
- 3 DB-backed add_playlist tests (with a mocked expander, so no
network): count aggregation across new + pre-seeded duplicates,
bad-entry tolerance, and the empty-playlist case.
- 1 opt-in live-network test gated on SECOND_BRAIN_LIVE_NETWORK_TESTS=1
exercising expand_youtube_playlist against a real public playlist.
Live-verified end to end:
- web POST of a real 13-entry public playlist queued 13 video rows
with titles, flash showed "Queued 13 videos".
- re-POST returned "Queued 0 videos (13 duplicates skipped)".
- watch?v=…&list=… correctly stayed a single-add.
- CLI parity confirmed against the same playlist.
Extracts the `second-brain add` CLI's queueing logic into a service
module (sources_service.add_source) so the CLI and the new web POST
route share the same validation, dedupe, and source_type heuristic —
no behaviour drift between the two entry points.
UI:
- The dashboard body (Pipeline counts + Settings snapshot + Recent
activity) moves into _dashboard_body.html, wrapped in
`<div id="dashboard-body">`. HTMX targets that id for swap.
- A new "Add to queue" section sits at the top of the partial: URL
(required, type=url), Domain (select matching DOMAINS), Title and
Focus (both optional, match the CLI flags). hx-post=/sources/add,
hx-target=#dashboard-body, hx-swap=outerHTML — same pattern as the
settings save form.
Route:
- POST /sources/add calls sources_service.add_source inside a single
transaction, then re-renders _dashboard_body.html so the pipeline
counts and recent-activity list update in place. Flash slots above
the form report:
✓ Queued <type>: <url> on a fresh add
✓ Already queued (id=…, status=…) on a dupe (matches CLI text)
✗ <validation message> on bad URL / unknown domain
CLI:
- `second-brain add` now delegates to the same service. Field values
for the success echo are captured inside the session block so a
post-commit detached-instance access can't fail. Bad input exits 2
with a clear stderr message instead of raising.
Live-verified through the web container on herbys-dev: dashboard
renders the form, happy add lands a row, dup-detect matches the CLI
phrasing, two validation errors surface as red flashes, swap target
id survives across swaps, no tracebacks in uvicorn logs.
