second-brain/alembic/versions
Travis Herbranson d055d1798d transcripts: capture segment-level output into a new JSONB column
Tower worker now persists faster-whisper's segment-level output
(start/end/text + word-level timing when available) alongside the
existing joined `transcript_text`. The text column stays the canonical
input the extractor reads — this is additive.

Changes:

- alembic v4: sources.transcript_segments JSONB NULL. JSONB rather than
  JSON so future equality/containment queries are indexable without a
  re-migration. Same lovebug-no-CREATE-on-petalbrain guard as prior
  migrations.

- ORM model: Optional[list] mapped to JSONB (postgresql dialect).

- transcribe.py:
  - Always pass word_timestamps=True to faster-whisper.transcribe.
  - New segment_to_dict() flattens the upstream NamedTuple-shaped
    Segment/Word into JSON-safe plain dicts so the JSONB write doesn't
    drag faster-whisper into any reader.
  - Per-word defensive conversion: a single malformed word can't drop
    the surrounding segment.

- transcribe_worker._advance: after a successful transcribe, persist
  segments into source.transcript_segments inside a try/except. If the
  JSONB write fails (oversize row, malformed dict, etc.) we log a
  warning and still commit transcript_text + status=TRANSCRIBED — the
  pipeline never crashes over the additive index.

- Tests: three new unit tests against fake Segment/Word objects cover
  the happy path (word entries serialise), the no-words case
  (`segment.words is None` → empty list), and the malformed-word skip.
  json.dumps(d) asserts JSONB-binding compatibility.

Live-verified: migration applied clean against petalbrain (`\d sources`
shows transcript_segments jsonb); ORM round-trip writes and reads the
sample payload identically. GPU large-v3 word-timestamp behaviour is
unchanged from upstream — only the tower can validate that hot path.
2026-05-25 13:40:49 -04:00
..
4a1e2f6c9d10_v1_second_brain_pipeline_tables.py alembic: skip CREATE SCHEMA when present + add embedding smoke test 2026-05-24 22:49:06 -04:00
7b3e8a5c1f29_v2_pipeline_settings.py settings: v2 migration + ORM + settings_store helper 2026-05-25 08:11:53 -04:00
c8f1d9e34b7a_v3_source_claim_columns.py pipeline split: tower transcribe worker + claim queue + faster-whisper 2026-05-25 10:11:37 -04:00
d4a72f51c8e3_v4_sources_transcript_segments.py transcripts: capture segment-level output into a new JSONB column 2026-05-25 13:40:49 -04:00