d055d1798d
Tower worker now persists faster-whisper's segment-level output
(start/end/text + word-level timing when available) alongside the
existing joined `transcript_text`. The text column stays the canonical
input the extractor reads — this is additive.
Changes:
- alembic v4: sources.transcript_segments JSONB NULL. JSONB rather than
JSON so future equality/containment queries are indexable without a
re-migration. Same lovebug-no-CREATE-on-petalbrain guard as prior
migrations.
- ORM model: Optional[list] mapped to JSONB (postgresql dialect).
- transcribe.py:
- Always pass word_timestamps=True to faster-whisper.transcribe.
- New segment_to_dict() flattens the upstream NamedTuple-shaped
Segment/Word into JSON-safe plain dicts so the JSONB write doesn't
drag faster-whisper into any reader.
- Per-word defensive conversion: a single malformed word can't drop
the surrounding segment.
- transcribe_worker._advance: after a successful transcribe, persist
segments into source.transcript_segments inside a try/except. If the
JSONB write fails (oversize row, malformed dict, etc.) we log a
warning and still commit transcript_text + status=TRANSCRIBED — the
pipeline never crashes over the additive index.
- Tests: three new unit tests against fake Segment/Word objects cover
the happy path (word entries serialise), the no-words case
(`segment.words is None` → empty list), and the malformed-word skip.
json.dumps(d) asserts JSONB-binding compatibility.
Live-verified: migration applied clean against petalbrain (`\d sources`
shows transcript_segments jsonb); ORM round-trip writes and reads the
sample payload identically. GPU large-v3 word-timestamp behaviour is
unchanged from upstream — only the tower can validate that hot path.
43 lines
1.4 KiB
Python
43 lines
1.4 KiB
Python
"""v4 second_brain: sources.transcript_segments JSONB
|
|
|
|
Revision ID: d4a72f51c8e3
|
|
Revises: c8f1d9e34b7a
|
|
Create Date: 2026-05-25 13:00:00.000000
|
|
|
|
Adds a JSONB column to `sources` for the tower-side worker to persist
|
|
segment-level transcription output (start/end/text per segment + word-
|
|
level timing when faster-whisper returns it). Additive: `transcript_text`
|
|
remains the canonical full-text the extractor consumes; this column is a
|
|
secondary index for future use (chunked retrieval, timestamp-anchored
|
|
quoting, etc.).
|
|
|
|
JSONB rather than JSON so equality/containment ops are indexable later
|
|
without a re-migration. No index added yet — Travis flagged "no
|
|
gold-plating" and the column may sit unused for a while.
|
|
|
|
Same lovebug-no-CREATE-on-petalbrain guard as the prior migrations:
|
|
env.py has already bootstrapped the schema, this just SETs search_path
|
|
and emits DDL inside it.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from collections.abc import Sequence
|
|
|
|
from alembic import op
|
|
|
|
revision: str = "d4a72f51c8e3"
|
|
down_revision: str | Sequence[str] | None = "c8f1d9e34b7a"
|
|
branch_labels: str | Sequence[str] | None = None
|
|
depends_on: str | Sequence[str] | None = None
|
|
|
|
|
|
def upgrade() -> None:
|
|
op.execute("SET search_path TO second_brain")
|
|
op.execute("ALTER TABLE sources ADD COLUMN transcript_segments JSONB")
|
|
|
|
|
|
def downgrade() -> None:
|
|
op.execute("SET search_path TO second_brain")
|
|
op.execute("ALTER TABLE sources DROP COLUMN IF EXISTS transcript_segments")
|