Compare commits

..

No commits in common. "24055ad686c7de74fcd9f0cece6753516df0746c" and "373e137993610f790d823d0b2c495c04c1855226" have entirely different histories.

11 changed files with 133 additions and 431 deletions

View File

@ -15,31 +15,16 @@ and all extraction/embedding stay on the dev side.
# Build / runtime tooling. # Build / runtime tooling.
sudo pacman -S --needed base-devel git ffmpeg python uv sudo pacman -S --needed base-devel git ffmpeg python uv
# NVIDIA driver only — see the CUDA-12 gotcha below before installing # NVIDIA — driver, CUDA, cuDNN. faster-whisper / CTranslate2 needs cuDNN
# the distro's `cuda` / `cudnn` packages. # at runtime; missing-symbol errors at first transcribe are usually a
sudo pacman -S --needed nvidia nvidia-utils # cuDNN install gap. Use whichever driver/cuda channel matches your
# Arch derivative; on stock EndeavourOS the AUR/community packages work:
sudo pacman -S --needed nvidia nvidia-utils cuda cudnn
# Confirm the GPU is visible. # Confirm the GPU is visible.
nvidia-smi nvidia-smi
``` ```
### ⚠ CUDA-12 gotcha — do NOT install pacman's `cuda` / `cudnn`
Arch / EndeavourOS now ships **CUDA 13** (`libcublas.so.13`). The
`ctranslate2` 4.7.x backend that `faster-whisper` 1.x rides on top of
wants **CUDA 12** (`libcublas.so.12`) + **cuDNN 9** (`libcudnn.so.9`)
and will not load against the system's CUDA-13 libs.
Rather than fight the distro, this project pins the CUDA-12 runtime as
pip wheels in the venv via the `tower` extra (`nvidia-cublas-cu12`,
`nvidia-cudnn-cu12>=9,<10`). The `uv sync --extra tower` step in §2
brings them in automatically; the systemd `ExecStart` wrapper
(`deploy/tower/run-transcribe-worker.sh`) prepends those wheel-bundled
`.so` directories to `LD_LIBRARY_PATH` at start time.
If you've already installed pacman's `cuda` / `cudnn`, you can leave
them — the wrapper's `LD_LIBRARY_PATH` prepend wins for this service.
yt-dlp comes in via the Python project (`uv sync` step below), so the yt-dlp comes in via the Python project (`uv sync` step below), so the
distro yt-dlp is *optional*. Leave it off the host unless you want it on distro yt-dlp is *optional*. Leave it off the host unless you want it on
your CLI PATH. your CLI PATH.
@ -60,23 +45,11 @@ cd second-brain
uv sync --extra tower uv sync --extra tower
``` ```
`uv sync --extra tower` will pull ~1 GB of pinned CUDA-12 wheels `faster-whisper` will pull a couple hundred MB of CUDA wheels. First-run
(`nvidia-cublas-cu12`, `nvidia-cudnn-cu12`, plus `faster-whisper` / model download (~3 GB for large-v3) lands under `~/.cache/huggingface/`
`ctranslate2`). First-run model download (~3 GB for large-v3) lands the first time the worker transcribes; pre-warm it if you want:
under `~/.cache/huggingface/` the first time the worker transcribes;
pre-warm if you want:
```bash ```bash
# Run the wrapper so the CUDA-12 lib paths are pinned exactly the way
# the systemd service will see them — this is the cheapest end-to-end
# sanity check that the venv + LD_LIBRARY_PATH plumbing is correct.
deploy/tower/run-transcribe-worker.sh --once
```
Or just instantiate the model directly:
```bash
LD_LIBRARY_PATH="$(uv run python -c 'import os, nvidia.cublas, nvidia.cudnn; print(":".join(os.path.join(os.path.dirname(m.__file__),"lib") for m in [nvidia.cublas, nvidia.cudnn]))')" \
uv run python -c "from faster_whisper import WhisperModel; WhisperModel('large-v3', device='cuda', compute_type='float16')" uv run python -c "from faster_whisper import WhisperModel; WhisperModel('large-v3', device='cuda', compute_type='float16')"
``` ```
@ -107,41 +80,16 @@ configured separately as part of Travis's infra.
## 4. EnvironmentFile ## 4. EnvironmentFile
systemd reads this BEFORE dropping to `User=herbyadmin`. The file is
**not** a shell init script — `export VAR=…` in your shell does not
apply here, and changes require `systemctl restart` to take effect.
There's a ready-to-scp file on **herbys-dev** with the real lovebug
password already populated, generated from
`/opt/backups/postgres-consolidation/credentials.env`:
```
/opt/backups/postgres-consolidation/second-brain-transcribe.env (mode 0600)
```
Pull it across instead of hand-copying the password:
```bash ```bash
# from the tower sudo install -m 0600 -o root -g root \
scp herbys-dev:/opt/backups/postgres-consolidation/second-brain-transcribe.env /tmp/sb-env deploy/tower/second-brain-transcribe.env.example \
sudo install -m 0600 -o root -g root /tmp/sb-env /etc/default/second-brain-transcribe /etc/default/second-brain-transcribe
rm /tmp/sb-env sudo $EDITOR /etc/default/second-brain-transcribe # paste the lovebug password
``` ```
If you ever need to regenerate the dev-host file (e.g. after a password The lovebug password lives in `/opt/backups/postgres-consolidation/credentials.env`
rotation), the recipe is: on the dev host. Copy it across through whatever channel you trust
(personal Bitwarden, scp over WG, etc.) — do not commit it.
```bash
# on herbys-dev — never echoes the password, mode 0600
PW=$(grep ^LOVEBUG_PG_PASSWORD= /opt/backups/postgres-consolidation/credentials.env | cut -d= -f2-)
umask 077
echo "SECOND_BRAIN_DATABASE_URL=postgresql://lovebug:${PW}@db.wg.herbylab.dev:5432/petalbrain" \
> /opt/backups/postgres-consolidation/second-brain-transcribe.env
unset PW
```
The committed `deploy/tower/second-brain-transcribe.env.example`
documents the format and never carries a real secret.
--- ---
@ -159,12 +107,6 @@ the worker starts after the tunnel comes up at boot. If the tunnel
disappears mid-run the worker keeps polling and logs DB-unreachable disappears mid-run the worker keeps polling and logs DB-unreachable
warnings; it doesn't crash-loop. warnings; it doesn't crash-loop.
`ExecStart=` calls the in-repo `deploy/tower/run-transcribe-worker.sh`
wrapper, which resolves the venv's CUDA-12 lib dirs at runtime and
prepends them to `LD_LIBRARY_PATH` before exec-ing `uv run`. systemd
does NOT inherit your shell's `LD_LIBRARY_PATH`, so the wrapper is
how the CUDA-12 pin survives across reboots and python upgrades.
### Day-to-day commands ### Day-to-day commands
```bash ```bash

View File

@ -1,64 +0,0 @@
#!/usr/bin/env bash
#
# Tower-side launcher for the transcribe worker.
#
# CTranslate2 4.7.2 needs libcublas.so.12 + libcudnn.so.9. Arch/EndeavourOS
# now ships CUDA 13, so we install the matching CUDA-12 runtime as pip
# wheels under the `tower` extra (nvidia-cublas-cu12, nvidia-cudnn-cu12).
# Those wheels drop their .so files into the venv's
# `site-packages/nvidia/{cublas,cudnn}/lib/` dirs, but nothing exports
# those onto LD_LIBRARY_PATH — systemd doesn't inherit Travis's shell.
#
# This wrapper resolves those paths at runtime from the venv's own
# Python (so it's robust to python-3.X upgrades, venv moves, and wheel
# version bumps), prepends them to LD_LIBRARY_PATH, and execs the
# worker. If the nvidia packages aren't importable we abort loudly so
# systemd surfaces a clear error instead of a silent libcublas load
# failure deep inside ctranslate2.
set -euo pipefail
cd "$(dirname "$(readlink -f "$0")")/../.." # project root: deploy/tower → ../..
# Resolve lib dirs from inside the project venv via `uv run`. Both
# packages must import cleanly — uv exits non-zero if not, and the
# `set -e` above propagates that.
NVIDIA_LIB_DIRS="$(
uv run python - <<'PY'
import os, sys
# nvidia.cublas / nvidia.cudnn are PEP 420 namespace packages — they
# have __path__ but no __init__.py and no __file__. Resolve via __path__
# (which is a _NamespacePath; treat as iterable of strings).
mods = ("nvidia.cublas", "nvidia.cudnn")
dirs = []
for m in mods:
try:
pkg = __import__(m, fromlist=["_"])
except ImportError as exc:
sys.stderr.write(
f"[run-transcribe-worker] missing CUDA-12 runtime package {m!r}: {exc}\n"
"Install with: uv sync --extra tower\n"
)
sys.exit(1)
paths = list(getattr(pkg, "__path__", []) or [])
if not paths:
sys.stderr.write(
f"[run-transcribe-worker] {m} has no __path__ — broken install?\n"
)
sys.exit(1)
lib = os.path.join(paths[0], "lib")
if not os.path.isdir(lib):
sys.stderr.write(
f"[run-transcribe-worker] {m} is installed but {lib!r} is not a directory\n"
)
sys.exit(1)
dirs.append(lib)
print(":".join(dirs))
PY
)"
# Prepend so our pinned CUDA-12 libs win over any system CUDA 13 left on
# the loader path. Preserve any existing LD_LIBRARY_PATH (rare but cheap).
export LD_LIBRARY_PATH="${NVIDIA_LIB_DIRS}${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}"
exec uv run second-brain transcribe-worker "$@"

View File

@ -1,38 +1,24 @@
# /etc/default/second-brain-transcribe (template — copy + fill in the password).
#
# systemd reads this BEFORE dropping to User=herbyadmin, so installing it
# root:root mode 0600 is the right shape. NEVER commit the real value.
#
# Sourcing the password:
# The lovebug Postgres password is the LOVEBUG_PG_PASSWORD value in
# /opt/backups/postgres-consolidation/credentials.env on herbys-dev
# (mode 0600, host-only). It's a 64-character hex string — no URL
# encoding required.
#
# On herbys-dev there's already a ready-to-scp env file with the real
# password populated:
# /opt/backups/postgres-consolidation/second-brain-transcribe.env
# scp that to the tower instead of hand-copying:
# scp herbys-dev:/opt/backups/postgres-consolidation/second-brain-transcribe.env \
# tower:/tmp/sb-env
# sudo install -m 0600 -o root -g root /tmp/sb-env \
# /etc/default/second-brain-transcribe # /etc/default/second-brain-transcribe
# rm /tmp/sb-env
# #
# DB host is db.wg.herbylab.dev:5432 over WireGuard, database name # Copy to /etc/default/second-brain-transcribe, chmod 0600, chown root:root
# `petalbrain` (the post-consolidation rename). # (systemd reads it before dropping to User=herbyadmin so a non-readable
# # permissions mode is fine — the herbyadmin user never touches this file).
# Note: this file is NOT a shell init script — `export VAR=...`
# additions in your interactive shell don't apply here, and changes
# require `systemctl restart second-brain-transcribe` to take effect.
SECOND_BRAIN_DATABASE_URL=postgresql://lovebug:REPLACE_WITH_LOVEBUG_PG_PASSWORD@db.wg.herbylab.dev:5432/petalbrain # REQUIRED — petalbrain Postgres reached over the WireGuard tunnel.
# Hub-side hostname is db.wg.herbylab.dev (10.99.0.1); the tower is 10.99.0.2.
# No TLS — WireGuard already encrypts, and lovebug doesn't require SSL.
SECOND_BRAIN_DATABASE_URL=postgresql+psycopg://lovebug:REPLACE_ME@db.wg.herbylab.dev:5432/petalbrain
# OPTIONAL — only set to override the [whisper] block in settings.toml. # OPTIONAL — only set if the tower has a non-standard whisper layout.
# Defaults are large-v3 / cuda / float16. # Defaults from settings.toml: model=large-v3, device=cuda, compute_type=float16.
# WHISPER_MODEL=large-v3 # WHISPER_MODEL=large-v3
# WHISPER_DEVICE=cuda # WHISPER_DEVICE=cuda
# WHISPER_COMPUTE_TYPE=float16 # WHISPER_COMPUTE_TYPE=float16
# OPTIONAL — identifier stamped into sources.claimed_by (default tower:<hostname>). # OPTIONAL — identifier stamped into sources.claimed_by (default tower:<hostname>).
# Useful if you ever run multiple tower workers and want to tell them apart.
# SECOND_BRAIN_WORKER_ID=tower-main # SECOND_BRAIN_WORKER_ID=tower-main
# OPTIONAL — bump worker log verbosity. INFO is the default.
# PYTHONUNBUFFERED=1
# SECOND_BRAIN_LOG_LEVEL=DEBUG

View File

@ -7,13 +7,6 @@ Documentation=https://gitea.plantbasedsoutherner.com/petal-power/second-brain
After=network-online.target wg-quick@wg-lan.service After=network-online.target wg-quick@wg-lan.service
Wants=network-online.target wg-quick@wg-lan.service Wants=network-online.target wg-quick@wg-lan.service
# Crash loop guard — systemd will give up after this if the unit keeps
# dying. The worker itself logs+backs-off on DB outages so this only
# trips on genuine failure. (StartLimit* directives live in [Unit] on
# modern systemd; placing them in [Service] is a silent no-op.)
StartLimitIntervalSec=300
StartLimitBurst=5
[Service] [Service]
Type=simple Type=simple
User=trucktrav User=trucktrav
@ -25,17 +18,20 @@ WorkingDirectory=/opt/projects/homelab/second-brain
# Env (SECOND_BRAIN_DATABASE_URL, optional knobs). 0600, owned by herbyadmin. # Env (SECOND_BRAIN_DATABASE_URL, optional knobs). 0600, owned by herbyadmin.
EnvironmentFile=/etc/default/second-brain-transcribe EnvironmentFile=/etc/default/second-brain-transcribe
# The wrapper resolves the venv's CUDA-12 lib dirs (nvidia-cublas-cu12 / # `uv run` resolves the project venv. Keep the worker in the foreground so
# nvidia-cudnn-cu12 pulled in by `uv sync --extra tower`), prepends them # systemd owns the lifecycle; the worker handles SIGTERM/SIGINT cleanly and
# to LD_LIBRARY_PATH, then execs the worker via `uv run`. systemd does # exits after the current iteration finishes.
# NOT inherit a shell's LD_LIBRARY_PATH, so doing this inline in the ExecStart=/usr/bin/uv run second-brain transcribe-worker
# unit would require a brittle hard-coded `python3.XX` path and break
# on Python upgrades.
ExecStart=/opt/projects/homelab/second-brain/deploy/tower/run-transcribe-worker.sh
Restart=always Restart=always
RestartSec=10 RestartSec=10
# Crash loop guard — systemd will give up after this if the unit keeps
# dying. The worker itself logs+backs-off on DB outages so this only
# trips on genuine failure.
StartLimitIntervalSec=300
StartLimitBurst=5
# A few sandboxing nice-to-haves. Loose on purpose — the worker needs # A few sandboxing nice-to-haves. Loose on purpose — the worker needs
# /dev/nvidia* for CUDA and writes to user-owned media/subtitles dirs. # /dev/nvidia* for CUDA and writes to user-owned media/subtitles dirs.
ProtectSystem=full ProtectSystem=full

View File

@ -28,19 +28,10 @@ dependencies = [
# Max OAuth subscription and has no Python-side Anthropic dependency. # Max OAuth subscription and has no Python-side Anthropic dependency.
api = ["anthropic>=0.40.0"] api = ["anthropic>=0.40.0"]
# Tower-only: faster-whisper (CTranslate2) + the matched CUDA-12 runtime # Tower-only: faster-whisper (CTranslate2) for the transcribe worker.
# pulled in as wheels. Arch/EndeavourOS now ships CUDA 13 (libcublas.so.13) # Pulls in CUDA wheels — install with `uv sync --extra tower` on the
# which ctranslate2 4.7+ cannot load — it wants libcublas.so.12 + cuDNN 9. # tower box, leave it off on dev. CPU/int8 path works on this same wheel.
# Pinning the CUDA-12 runtime via pip wheels in the venv sidesteps the tower = ["faster-whisper>=1.0.0"]
# system CUDA mismatch entirely. sys_platform marker keeps these out of
# any non-Linux resolve (the nvidia wheels are Linux-only anyway, but the
# marker makes that explicit). Install with `uv sync --extra tower`; do
# not install on dev.
tower = [
"faster-whisper>=1.0.0",
"nvidia-cublas-cu12; sys_platform == 'linux'",
"nvidia-cudnn-cu12>=9,<10; sys_platform == 'linux'",
]
[dependency-groups] [dependency-groups]
dev = [ dev = [

View File

@ -141,41 +141,32 @@ async def source_detail(request: Request, source_id: int):
) )
_IN_FLIGHT_STATUSES = [ @app.get("/queue", response_class=HTMLResponse)
SourceStatus.PENDING, async def queue_view(request: Request):
SourceStatus.PULLED, """Queue view: pending and in-flight sources."""
SourceStatus.TRANSCRIBED, db = get_database()
] in_flight_statuses = [SourceStatus.PENDING, SourceStatus.PULLED, SourceStatus.TRANSCRIBED]
with db.session() as sess:
def _build_queue_context(
sess,
*,
add_flash: Optional[str] = None,
add_error: Optional[str] = None,
) -> dict:
"""Context for queue.html / _queue_body.html (in-flight source list + form)."""
sources = ( sources = (
sess.query(Source) sess.query(Source)
.filter(Source.status.in_(_IN_FLIGHT_STATUSES)) .filter(Source.status.in_(in_flight_statuses))
.order_by(Source.ingested_at.asc()) .order_by(Source.ingested_at.asc())
.all() .all()
) )
return { source_list = [_source_to_dict(s) for s in sources]
"sources": [_source_to_dict(s) for s in sources],
return templates.TemplateResponse(
request,
"index.html",
{
"sources": source_list,
"domains": DOMAINS, "domains": DOMAINS,
"add_flash": add_flash, "statuses": [s.value for s in SourceStatus],
"add_error": add_error, "selected_domain": "",
} "selected_status": "",
"page_title": "Queue",
},
@app.get("/queue", response_class=HTMLResponse) )
async def queue_view(request: Request):
"""Queue view: pending and in-flight sources, with an add-to-queue form."""
db = get_database()
with db.session() as sess:
ctx = _build_queue_context(sess)
return templates.TemplateResponse(request, "queue.html", ctx)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@ -432,31 +423,12 @@ async def sources_add(
except ValueError as exc: except ValueError as exc:
add_error = str(exc) add_error = str(exc)
# Dispatch the response partial based on which page submitted the form.
# HTMX always sends HX-Current-URL with the request URL of the user's
# current page; we parse the path and pick the matching body partial.
# Same /sources/add endpoint, same service call — only the rendered
# region differs, so each page refreshes its own list/counts inline.
from urllib.parse import urlparse as _urlparse
hx_url = request.headers.get("HX-Current-URL") or request.headers.get(
"Referer", ""
)
source_path = _urlparse(hx_url).path if hx_url else ""
with db.session() as sess: with db.session() as sess:
if source_path.rstrip("/") == "/queue":
ctx = _build_queue_context(
sess, add_flash=add_flash, add_error=add_error
)
template_name = "_queue_body.html"
else:
ctx = _build_dashboard_context( ctx = _build_dashboard_context(
sess, add_flash=add_flash, add_error=add_error sess, add_flash=add_flash, add_error=add_error
) )
template_name = "_dashboard_body.html"
return templates.TemplateResponse(request, template_name, ctx) return templates.TemplateResponse(request, "_dashboard_body.html", ctx)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------

View File

@ -1,67 +0,0 @@
{# Add-to-queue form. Reused by /dashboard and /queue — the only
difference between the two callers is the HTMX swap target, which the
parent template sets via `{% set swap_target = "#…-body" %}` before
including this partial. POST goes to /sources/add either way; the
route inspects HX-Current-URL to pick the right body partial in its
response. #}
<section class="bg-white rounded-lg shadow-sm p-5 mb-6">
<div class="flex items-center justify-between mb-3">
<h3 class="text-sm font-semibold uppercase tracking-wide text-gray-500">
Add to queue
</h3>
<p class="text-xs text-gray-400">video on a known host (YouTube / Vimeo) → tower; otherwise treated as an article. YouTube /playlist?list=… fans out.</p>
</div>
{% if add_flash %}
<div class="mb-3 px-3 py-2 rounded bg-green-50 border border-green-200 text-green-700 text-sm">
✓ {{ add_flash }}
</div>
{% endif %}
{% if add_error %}
<div class="mb-3 px-3 py-2 rounded bg-red-50 border border-red-200 text-red-700 text-sm">
✗ {{ add_error }}
</div>
{% endif %}
<form hx-post="/sources/add"
hx-target="{{ swap_target }}"
hx-swap="outerHTML"
class="grid grid-cols-1 md:grid-cols-12 gap-3 items-end">
<div class="md:col-span-6">
<label class="block text-xs font-medium text-gray-600 mb-1">URL</label>
<input type="url" name="url" required autofocus
placeholder="https://…"
class="text-sm border border-gray-300 rounded px-3 py-1.5 focus:outline-none focus:ring-2 focus:ring-indigo-400 w-full">
</div>
<div class="md:col-span-2">
<label class="block text-xs font-medium text-gray-600 mb-1">Domain</label>
<select name="domain"
class="text-sm border border-gray-300 rounded px-3 py-1.5 focus:outline-none focus:ring-2 focus:ring-indigo-400 w-full">
{% for d in domains %}
<option value="{{ d }}" {% if d == "development" %}selected{% endif %}>{{ d }}</option>
{% endfor %}
</select>
</div>
<div class="md:col-span-3">
<label class="block text-xs font-medium text-gray-600 mb-1">Title <span class="text-gray-400">(optional)</span></label>
<input type="text" name="title"
class="text-sm border border-gray-300 rounded px-3 py-1.5 focus:outline-none focus:ring-2 focus:ring-indigo-400 w-full">
</div>
<div class="md:col-span-1">
<button type="submit"
class="w-full px-4 py-1.5 bg-indigo-600 text-white text-sm font-medium rounded hover:bg-indigo-700 transition-colors">
Add
</button>
</div>
<div class="md:col-span-12">
<label class="block text-xs font-medium text-gray-600 mb-1">Focus <span class="text-gray-400">(optional — narrows the extraction prompt)</span></label>
<input type="text" name="focus"
class="text-sm border border-gray-300 rounded px-3 py-1.5 focus:outline-none focus:ring-2 focus:ring-indigo-400 w-full">
</div>
</form>
</section>

View File

@ -18,8 +18,68 @@
} %} } %}
<div id="dashboard-body"> <div id="dashboard-body">
{% set swap_target = "#dashboard-body" %} <!-- Add source -->
{% include "_add_to_queue_form.html" %} <section class="bg-white rounded-lg shadow-sm p-5 mb-6">
<div class="flex items-center justify-between mb-3">
<h3 class="text-sm font-semibold uppercase tracking-wide text-gray-500">
Add to queue
</h3>
<p class="text-xs text-gray-400">video on a known host (YouTube / Vimeo) → tower; otherwise treated as an article</p>
</div>
{% if add_flash %}
<div class="mb-3 px-3 py-2 rounded bg-green-50 border border-green-200 text-green-700 text-sm">
✓ {{ add_flash }}
</div>
{% endif %}
{% if add_error %}
<div class="mb-3 px-3 py-2 rounded bg-red-50 border border-red-200 text-red-700 text-sm">
✗ {{ add_error }}
</div>
{% endif %}
<form hx-post="/sources/add"
hx-target="#dashboard-body"
hx-swap="outerHTML"
class="grid grid-cols-1 md:grid-cols-12 gap-3 items-end">
<div class="md:col-span-6">
<label class="block text-xs font-medium text-gray-600 mb-1">URL</label>
<input type="url" name="url" required autofocus
placeholder="https://…"
class="text-sm border border-gray-300 rounded px-3 py-1.5 focus:outline-none focus:ring-2 focus:ring-indigo-400 w-full">
</div>
<div class="md:col-span-2">
<label class="block text-xs font-medium text-gray-600 mb-1">Domain</label>
<select name="domain"
class="text-sm border border-gray-300 rounded px-3 py-1.5 focus:outline-none focus:ring-2 focus:ring-indigo-400 w-full">
{% for d in domains %}
<option value="{{ d }}" {% if d == "development" %}selected{% endif %}>{{ d }}</option>
{% endfor %}
</select>
</div>
<div class="md:col-span-3">
<label class="block text-xs font-medium text-gray-600 mb-1">Title <span class="text-gray-400">(optional)</span></label>
<input type="text" name="title"
class="text-sm border border-gray-300 rounded px-3 py-1.5 focus:outline-none focus:ring-2 focus:ring-indigo-400 w-full">
</div>
<div class="md:col-span-1">
<button type="submit"
class="w-full px-4 py-1.5 bg-indigo-600 text-white text-sm font-medium rounded hover:bg-indigo-700 transition-colors">
Add
</button>
</div>
<div class="md:col-span-12">
<label class="block text-xs font-medium text-gray-600 mb-1">Focus <span class="text-gray-400">(optional — narrows the extraction prompt)</span></label>
<input type="text" name="focus"
class="text-sm border border-gray-300 rounded px-3 py-1.5 focus:outline-none focus:ring-2 focus:ring-indigo-400 w-full">
</div>
</form>
</section>
<!-- Pipeline overview --> <!-- Pipeline overview -->
<section class="bg-white rounded-lg shadow-sm p-5 mb-6"> <section class="bg-white rounded-lg shadow-sm p-5 mb-6">

View File

@ -1,62 +0,0 @@
{# HTMX-replaceable queue body. POST /sources/add re-renders this when
the form was submitted from /queue (dispatched server-side via
HX-Current-URL); hx-target="#queue-body" + hx-swap="outerHTML" keeps
the wrapping div in place for the next swap. #}
{% set status_colors = {
'pending': 'bg-gray-100 text-gray-700',
'pulled': 'bg-blue-100 text-blue-700',
'transcribed': 'bg-yellow-100 text-yellow-700',
'analyzed': 'bg-orange-100 text-orange-700',
'accepted': 'bg-green-100 text-green-700',
'published': 'bg-purple-100 text-purple-700',
'failed': 'bg-red-100 text-red-700'
} %}
{% set domain_colors = {
'development': 'bg-cyan-50 text-cyan-700 border-cyan-200',
'content': 'bg-pink-50 text-pink-700 border-pink-200',
'business': 'bg-amber-50 text-amber-700 border-amber-200',
'homelab': 'bg-teal-50 text-teal-700 border-teal-200'
} %}
<div id="queue-body">
{% set swap_target = "#queue-body" %}
{% include "_add_to_queue_form.html" %}
<!-- In-flight source list (pending / pulled / transcribed) -->
{% if sources %}
<div class="grid gap-3">
{% for source in sources %}
<a href="/sources/{{ source.id }}"
class="block bg-white rounded-lg border border-gray-200 p-4 hover:border-indigo-400 hover:shadow-sm transition-all duration-150 group">
<div class="flex items-start justify-between gap-4">
<div class="flex-1 min-w-0">
<h3 class="font-semibold text-gray-900 group-hover:text-indigo-700 break-words">
{{ source.title }}
</h3>
<p class="text-xs text-gray-400 mt-0.5 break-all">{{ source.url }}</p>
{% if source.focus %}
<p class="text-xs text-gray-500 mt-1 italic break-words">Focus: {{ source.focus }}</p>
{% endif %}
</div>
<div class="flex flex-col items-end gap-1.5 flex-shrink-0">
<span class="text-xs px-2 py-0.5 rounded-full font-medium
{{ status_colors.get(source.status, 'bg-gray-100 text-gray-700') }}">
{{ source.status }}
</span>
<span class="text-xs px-2 py-0.5 rounded border font-medium
{{ domain_colors.get(source.domain, 'bg-gray-50 text-gray-600 border-gray-200') }}">
{{ source.domain }}
</span>
<span class="text-xs text-gray-400">{{ source.ingested_at }}</span>
</div>
</div>
</a>
{% endfor %}
</div>
{% else %}
<div class="text-center py-20 bg-white rounded-lg border border-dashed border-gray-300">
<p class="text-gray-400 text-lg">Queue is empty.</p>
<p class="text-gray-400 text-sm mt-1">Add a URL above to get started.</p>
</div>
{% endif %}
</div>

View File

@ -1,15 +0,0 @@
{% extends "base.html" %}
{% block title %}Queue — second-brain{% endblock %}
{% block content %}
<div class="mb-6 flex items-center justify-between">
<h2 class="text-2xl font-semibold text-gray-800">
Queue
<span class="text-base font-normal text-gray-500 ml-2">({{ sources|length }})</span>
</h2>
<span class="text-sm text-gray-500">pending → pulled → transcribed</span>
</div>
{% include "_queue_body.html" %}
{% endblock %}

37
uv.lock generated
View File

@ -988,39 +988,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" }, { url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" },
] ]
[[package]]
name = "nvidia-cublas-cu12"
version = "12.9.2.10"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-cuda-nvrtc-cu12" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/f7/a2/c96163a0fff1839c0c9548bbdeae7b853b867009e33b9b9264adc238b1cf/nvidia_cublas_cu12-12.9.2.10-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:5572131a59c3eebeeb1c4c8144f772d49372c20124916e072a0e3fc30df421d5", size = 575012079, upload-time = "2026-04-08T18:51:47.303Z" },
{ url = "https://files.pythonhosted.org/packages/cb/c0/0a517bfe63ccd3b92eb254d264e28fca3c7cab75d07daea315250fb1bf73/nvidia_cublas_cu12-12.9.2.10-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:e4f53a8ca8c5d6e8c492d0d0a3d565ecb59a751b19cfdaa4f6da0ab2104c1702", size = 581240110, upload-time = "2026-04-08T18:52:31.532Z" },
]
[[package]]
name = "nvidia-cuda-nvrtc-cu12"
version = "12.9.86"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b8/85/e4af82cc9202023862090bfca4ea827d533329e925c758f0cde964cb54b7/nvidia_cuda_nvrtc_cu12-12.9.86-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:210cf05005a447e29214e9ce50851e83fc5f4358df8b453155d5e1918094dcb4", size = 89568129, upload-time = "2025-06-05T20:02:41.973Z" },
{ url = "https://files.pythonhosted.org/packages/64/eb/c2295044b8f3b3b08860e2f6a912b702fc92568a167259df5dddb78f325e/nvidia_cuda_nvrtc_cu12-12.9.86-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:096d4de6bda726415dfaf3198d4f5c522b8e70139c97feef5cd2ca6d4cd9cead", size = 44528905, upload-time = "2025-06-05T20:02:29.754Z" },
]
[[package]]
name = "nvidia-cudnn-cu12"
version = "9.22.0.52"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-cublas-cu12" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/51/ff/c6a098c1e0bccc68aac5f1684526cf8936abb7024dcc46dca315b8a6f47f/nvidia_cudnn_cu12-9.22.0.52-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:cd9011812498376f866b9826331cc965ac922eb2df8549c9f2989f10255e5001", size = 774536507, upload-time = "2026-05-08T15:36:09.067Z" },
{ url = "https://files.pythonhosted.org/packages/a0/8f/2ede6b758b7524608472010f632bdd3370ea271d715d1d66044614b84cdc/nvidia_cudnn_cu12-9.22.0.52-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:391b9a7ee6386daaca7f8dca41e83c2c99f760c9581a0400755e87b4287b8847", size = 718382818, upload-time = "2026-05-08T15:37:38.061Z" },
]
[[package]] [[package]]
name = "onnxruntime" name = "onnxruntime"
version = "1.26.0" version = "1.26.0"
@ -1540,8 +1507,6 @@ api = [
] ]
tower = [ tower = [
{ name = "faster-whisper" }, { name = "faster-whisper" },
{ name = "nvidia-cublas-cu12", marker = "sys_platform == 'linux'" },
{ name = "nvidia-cudnn-cu12", marker = "sys_platform == 'linux'" },
] ]
[package.dev-dependencies] [package.dev-dependencies]
@ -1561,8 +1526,6 @@ requires-dist = [
{ name = "faster-whisper", marker = "extra == 'tower'", specifier = ">=1.0.0" }, { name = "faster-whisper", marker = "extra == 'tower'", specifier = ">=1.0.0" },
{ name = "httpx", specifier = ">=0.28.0" }, { name = "httpx", specifier = ">=0.28.0" },
{ name = "jinja2", specifier = ">=3.1.0" }, { name = "jinja2", specifier = ">=3.1.0" },
{ name = "nvidia-cublas-cu12", marker = "sys_platform == 'linux' and extra == 'tower'" },
{ name = "nvidia-cudnn-cu12", marker = "sys_platform == 'linux' and extra == 'tower'", specifier = ">=9,<10" },
{ name = "psycopg", extras = ["binary"], specifier = ">=3.2" }, { name = "psycopg", extras = ["binary"], specifier = ">=3.2" },
{ name = "psycopg-pool", specifier = ">=3.2" }, { name = "psycopg-pool", specifier = ">=3.2" },
{ name = "pydantic", specifier = ">=2.0.0" }, { name = "pydantic", specifier = ">=2.0.0" },