Compare commits
No commits in common. "8d29f2fb088837bc08aac7e09b2b239c0e2347f0" and "93f71ebb14b90723fea9c0cacc66db805d601150" have entirely different histories.
8d29f2fb08
...
93f71ebb14
@ -1,37 +0,0 @@
|
|||||||
# a-review — 2026-05-25
|
|
||||||
|
|
||||||
**Model:** gemini
|
|
||||||
**Diff base:** `HEAD~1` (5 files changed, 615 insertions(+), 30 deletions(-))
|
|
||||||
**Session reference:** none
|
|
||||||
**Context:** CLAUDE.md + session notes + expanded diff
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
Excellent work. The changes are well-structured, follow existing project conventions, and include a comprehensive test suite. This is a high-quality contribution. I have only one minor finding and one informational note.
|
|
||||||
|
|
||||||
### [WARNING] Logic Error
|
|
||||||
|
|
||||||
* **File:** `src/second_brain/web/templates/source_detail.html`
|
|
||||||
* **Location:** Line 151
|
|
||||||
* **Issue:** The "Re-extract only" button is conditionally rendered with `{% if source.has_transcript and source.source_type == 'video' %}`. This is too restrictive. The backend service function `reset_source_for_retry` correctly allows an "extract" retry for any source that has content in `transcript_text`, which includes processed articles. Limiting this feature to only videos in the UI prevents a valid and useful action for non-video sources.
|
|
||||||
* **Suggestion:** Change the conditional to `{% if source.has_transcript %}`. This will correctly show the button for any source (article or video) that has been pulled and has content ready for extraction, which matches the backend's capability.
|
|
||||||
|
|
||||||
```html
|
|
||||||
<!-- Change this -->
|
|
||||||
{% if source.has_transcript and source.source_type == 'video' %}
|
|
||||||
|
|
||||||
<!-- To this -->
|
|
||||||
{% if source.has_transcript %}
|
|
||||||
```
|
|
||||||
|
|
||||||
### [INFO] Security
|
|
||||||
|
|
||||||
* **Files:** `src/second_brain/web/app.py`, `src/second_brain/web/templates/source_detail.html`
|
|
||||||
* **Observation:** The changes introduce new endpoints (`/edit`, `/retry`) that modify database state. According to `CLAUDE.md`, the application has "No auth" and relies on being deployed in a trusted, LAN-only environment. These new endpoints increase the number of ways an unauthorized user on the network could alter data.
|
|
||||||
* **Context:** This is not a new vulnerability, but an expansion of the existing trust model. As long as the application remains on a trusted network, this is acceptable per the project's documented security posture. It's worth keeping in mind that no CSRF protection is being used for these HTMX `POST` actions.
|
|
||||||
|
|
||||||
### Summary
|
|
||||||
|
|
||||||
The overall code quality is very high. The changes are thoughtfully implemented, with a clean separation between the CLI, the web layer, and a shared service layer. The refactoring in `web/app.py` to create `_render_source_detail` is a good move for maintainability. The addition of a thorough, live-DB test suite for the new service functions is commendable and demonstrates a commitment to quality.
|
|
||||||
|
|
||||||
Once the minor template logic issue is resolved, this change will be in perfect shape.
|
|
||||||
@ -85,28 +85,6 @@ def _merge(default: dict, override: dict) -> dict:
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def _overlay_sections(base: dict, overlay: dict) -> dict:
|
|
||||||
"""Section-level overlay: one level deep.
|
|
||||||
|
|
||||||
For each key in `overlay`, if both `base[key]` and `overlay[key]` are
|
|
||||||
dicts, merge their keys (overlay wins). Otherwise the overlay value
|
|
||||||
replaces the base value outright. Sections present in only one of
|
|
||||||
the two pass through untouched.
|
|
||||||
|
|
||||||
Use this — not `_merge` — when combining `settings.toml` with
|
|
||||||
`settings.local.toml`, so a local `[database] url = "..."` doesn't
|
|
||||||
silently wipe other `[database]` keys defined in the base file.
|
|
||||||
"""
|
|
||||||
out = dict(base)
|
|
||||||
for key, ov_val in (overlay or {}).items():
|
|
||||||
base_val = out.get(key)
|
|
||||||
if isinstance(base_val, dict) and isinstance(ov_val, dict):
|
|
||||||
out[key] = {**base_val, **ov_val}
|
|
||||||
else:
|
|
||||||
out[key] = ov_val
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
class Config:
|
class Config:
|
||||||
"""Holds resolved configuration values."""
|
"""Holds resolved configuration values."""
|
||||||
|
|
||||||
@ -197,24 +175,7 @@ _config_instance: Config | None = None
|
|||||||
|
|
||||||
|
|
||||||
def load_config(path: Path | None = None) -> Config:
|
def load_config(path: Path | None = None) -> Config:
|
||||||
"""Load (and cache) configuration from a TOML file.
|
"""Load (and cache) configuration from a TOML file."""
|
||||||
|
|
||||||
After loading the base file, look for a sibling `settings.local.toml`
|
|
||||||
and overlay it (section-level merge — see `_overlay_sections`). The
|
|
||||||
local file is gitignored and intended for host-specific secrets
|
|
||||||
(e.g. `[database].url` for a dev shell) without forking the tracked
|
|
||||||
base.
|
|
||||||
|
|
||||||
Path resolution:
|
|
||||||
- explicit `path=` arg wins
|
|
||||||
- else `SECOND_BRAIN_CONFIG` env var
|
|
||||||
- else walks up from CWD for `config/settings.toml` or `settings.toml`
|
|
||||||
|
|
||||||
The local-overlay sibling is looked up next to whichever base file
|
|
||||||
was chosen, so `SECOND_BRAIN_CONFIG=/etc/sb/settings.toml` will also
|
|
||||||
pick up `/etc/sb/settings.local.toml` if present. If no base file is
|
|
||||||
found at all, no overlay is attempted.
|
|
||||||
"""
|
|
||||||
global _config_instance
|
global _config_instance
|
||||||
if _config_instance is not None:
|
if _config_instance is not None:
|
||||||
return _config_instance
|
return _config_instance
|
||||||
@ -236,15 +197,6 @@ def load_config(path: Path | None = None) -> Config:
|
|||||||
with open(path, "rb") as fh:
|
with open(path, "rb") as fh:
|
||||||
raw = tomllib.load(fh)
|
raw = tomllib.load(fh)
|
||||||
|
|
||||||
# Section-level overlay from a sibling settings.local.toml.
|
|
||||||
# Filename is the one already covered in .gitignore — host
|
|
||||||
# secrets live here and never get committed.
|
|
||||||
local_path = path.parent / "settings.local.toml"
|
|
||||||
if local_path.exists():
|
|
||||||
with open(local_path, "rb") as fh:
|
|
||||||
local_raw = tomllib.load(fh)
|
|
||||||
raw = _overlay_sections(raw, local_raw)
|
|
||||||
|
|
||||||
_config_instance = Config(raw)
|
_config_instance = Config(raw)
|
||||||
return _config_instance
|
return _config_instance
|
||||||
|
|
||||||
|
|||||||
@ -4,7 +4,6 @@ second-brain CLI entry point.
|
|||||||
Commands:
|
Commands:
|
||||||
add — queue a source URL (video or article)
|
add — queue a source URL (video or article)
|
||||||
process — run the full pipeline on pending/pulled/transcribed sources
|
process — run the full pipeline on pending/pulled/transcribed sources
|
||||||
retry — reset a source so the pipeline re-processes it
|
|
||||||
serve — launch the FastAPI web UI
|
serve — launch the FastAPI web UI
|
||||||
compile — run the wiki compiler on accepted sources
|
compile — run the wiki compiler on accepted sources
|
||||||
schedule — start the overnight scheduler
|
schedule — start the overnight scheduler
|
||||||
@ -308,47 +307,6 @@ def serve(host: str, port: int, reload: bool) -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# retry
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@cli.command()
|
|
||||||
@click.argument("source_id", type=int)
|
|
||||||
@click.option(
|
|
||||||
"--mode",
|
|
||||||
type=click.Choice(["full", "extract"]),
|
|
||||||
default="full",
|
|
||||||
show_default=True,
|
|
||||||
help="`full` resets to PENDING (re-pull + re-process); "
|
|
||||||
"`extract` resets to TRANSCRIBED (requires existing transcript).",
|
|
||||||
)
|
|
||||||
def retry(source_id: int, mode: str) -> None:
|
|
||||||
"""Reset a source so the pipeline re-processes it.
|
|
||||||
|
|
||||||
Clears the claim pair and error_message and rewinds status to the
|
|
||||||
appropriate stage. Parity with the dashboard re-run buttons.
|
|
||||||
"""
|
|
||||||
from second_brain.sources_service import reset_source_for_retry
|
|
||||||
|
|
||||||
db = get_database()
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
source = reset_source_for_retry(
|
|
||||||
sess, source_id=source_id, mode=mode
|
|
||||||
)
|
|
||||||
new_status = source.status.value
|
|
||||||
url = source.url
|
|
||||||
except LookupError as exc:
|
|
||||||
click.echo(f"[retry] {exc}", err=True)
|
|
||||||
sys.exit(2)
|
|
||||||
except ValueError as exc:
|
|
||||||
click.echo(f"[retry] {exc}", err=True)
|
|
||||||
sys.exit(2)
|
|
||||||
|
|
||||||
click.echo(f"[retry] id={source_id} status → {new_status} ({url})")
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# compile
|
# compile
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
@ -275,126 +275,14 @@ def add_playlist(
|
|||||||
return result
|
return result
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Edit + retry helpers (shared by web routes and CLI)
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
# Fields the edit-mode form is allowed to mutate. URL is intentionally
|
|
||||||
# excluded — it's the UNIQUE dedupe key and editing it risks collisions
|
|
||||||
# plus breaks the embeddings 4-tuple identity (source_id is stable but
|
|
||||||
# semantic identity drifts).
|
|
||||||
EDITABLE_METADATA_FIELDS = ("domain", "title", "focus")
|
|
||||||
|
|
||||||
|
|
||||||
def update_source_metadata(
|
|
||||||
sess: Session,
|
|
||||||
*,
|
|
||||||
source_id: int,
|
|
||||||
domain: Optional[str] = None,
|
|
||||||
title: Optional[str] = None,
|
|
||||||
focus: Optional[str] = None,
|
|
||||||
) -> Source:
|
|
||||||
"""Update editable metadata on a source row.
|
|
||||||
|
|
||||||
Empty strings collapse to NULL for title/focus (matches `add_source`
|
|
||||||
semantics). Domain is required to stay in the configured set —
|
|
||||||
bumping the constraint is a separate decision.
|
|
||||||
|
|
||||||
Raises ValueError on validation failures and LookupError if the row
|
|
||||||
is missing. Caller owns the transaction boundary.
|
|
||||||
"""
|
|
||||||
source = sess.query(Source).filter(Source.id == source_id).first()
|
|
||||||
if source is None:
|
|
||||||
raise LookupError(f"source id={source_id} not found")
|
|
||||||
|
|
||||||
if domain is not None:
|
|
||||||
if domain not in DOMAINS:
|
|
||||||
raise ValueError(
|
|
||||||
f"unknown domain {domain!r}; valid: {', '.join(DOMAINS)}"
|
|
||||||
)
|
|
||||||
source.domain = domain
|
|
||||||
|
|
||||||
if title is not None:
|
|
||||||
source.title = title.strip() or None
|
|
||||||
|
|
||||||
if focus is not None:
|
|
||||||
source.focus = focus.strip() or None
|
|
||||||
|
|
||||||
source.updated_at = utcnow()
|
|
||||||
sess.flush()
|
|
||||||
return source
|
|
||||||
|
|
||||||
|
|
||||||
# Retry modes — keep the surface small. Anything else is a separate ask.
|
|
||||||
RETRY_FULL = "full"
|
|
||||||
RETRY_EXTRACT = "extract"
|
|
||||||
RETRY_MODES = (RETRY_FULL, RETRY_EXTRACT)
|
|
||||||
|
|
||||||
|
|
||||||
def reset_source_for_retry(
|
|
||||||
sess: Session,
|
|
||||||
*,
|
|
||||||
source_id: int,
|
|
||||||
mode: str = RETRY_FULL,
|
|
||||||
) -> Source:
|
|
||||||
"""Reset a source so the pipeline re-processes it.
|
|
||||||
|
|
||||||
Modes:
|
|
||||||
- "full": status → PENDING. Article pull or video download +
|
|
||||||
transcribe will happen again on the appropriate worker.
|
|
||||||
- "extract": status → TRANSCRIBED. Requires the row to already
|
|
||||||
carry a transcript (transcript_text). The dev scheduler will
|
|
||||||
re-run extraction without re-pulling the source. Cheap.
|
|
||||||
|
|
||||||
Always clears `claimed_by` / `claimed_at` / `error_message` so the
|
|
||||||
queue dance can pick it up cleanly and stale errors don't bleed
|
|
||||||
into the UI.
|
|
||||||
|
|
||||||
Raises ValueError on bad mode / missing transcript for extract mode,
|
|
||||||
and LookupError if the row is missing.
|
|
||||||
"""
|
|
||||||
if mode not in RETRY_MODES:
|
|
||||||
raise ValueError(
|
|
||||||
f"unknown retry mode {mode!r}; valid: {', '.join(RETRY_MODES)}"
|
|
||||||
)
|
|
||||||
|
|
||||||
source = sess.query(Source).filter(Source.id == source_id).first()
|
|
||||||
if source is None:
|
|
||||||
raise LookupError(f"source id={source_id} not found")
|
|
||||||
|
|
||||||
if mode == RETRY_EXTRACT:
|
|
||||||
if not (source.transcript_text and source.transcript_text.strip()):
|
|
||||||
raise ValueError(
|
|
||||||
"extract-only retry requires an existing transcript; "
|
|
||||||
"use full retry instead"
|
|
||||||
)
|
|
||||||
source.status = SourceStatus.TRANSCRIBED
|
|
||||||
else:
|
|
||||||
source.status = SourceStatus.PENDING
|
|
||||||
|
|
||||||
source.claimed_by = None
|
|
||||||
source.claimed_at = None
|
|
||||||
source.error_message = None
|
|
||||||
source.updated_at = utcnow()
|
|
||||||
sess.flush()
|
|
||||||
return source
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"AddResult",
|
"AddResult",
|
||||||
"DEFAULT_PLAYLIST_MAX_ITEMS",
|
"DEFAULT_PLAYLIST_MAX_ITEMS",
|
||||||
"EDITABLE_METADATA_FIELDS",
|
|
||||||
"PlaylistAddResult",
|
"PlaylistAddResult",
|
||||||
"RETRY_EXTRACT",
|
|
||||||
"RETRY_FULL",
|
|
||||||
"RETRY_MODES",
|
|
||||||
"add_playlist",
|
"add_playlist",
|
||||||
"add_source",
|
"add_source",
|
||||||
"expand_youtube_playlist",
|
"expand_youtube_playlist",
|
||||||
"is_article_url",
|
"is_article_url",
|
||||||
"is_youtube_playlist_url",
|
"is_youtube_playlist_url",
|
||||||
"reset_source_for_retry",
|
|
||||||
"update_source_metadata",
|
|
||||||
"validate_url",
|
"validate_url",
|
||||||
]
|
]
|
||||||
|
|||||||
@ -31,14 +31,9 @@ from second_brain.settings_store import (
|
|||||||
)
|
)
|
||||||
from second_brain.sources_service import (
|
from second_brain.sources_service import (
|
||||||
DEFAULT_PLAYLIST_MAX_ITEMS,
|
DEFAULT_PLAYLIST_MAX_ITEMS,
|
||||||
RETRY_EXTRACT,
|
|
||||||
RETRY_FULL,
|
|
||||||
RETRY_MODES,
|
|
||||||
add_playlist,
|
add_playlist,
|
||||||
add_source,
|
add_source,
|
||||||
is_youtube_playlist_url,
|
is_youtube_playlist_url,
|
||||||
reset_source_for_retry,
|
|
||||||
update_source_metadata,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@ -125,33 +120,16 @@ async def index(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _render_source_detail(
|
@app.get("/sources/{source_id}", response_class=HTMLResponse)
|
||||||
request: Request,
|
async def source_detail(request: Request, source_id: int):
|
||||||
source_id: int,
|
"""Source detail page with extraction results and accept/reject actions."""
|
||||||
*,
|
|
||||||
edit: bool = False,
|
|
||||||
flash: Optional[str] = None,
|
|
||||||
error: Optional[str] = None,
|
|
||||||
form_overrides: Optional[dict] = None,
|
|
||||||
) -> HTMLResponse:
|
|
||||||
"""Single render path for /sources/{id} — used by GET + every HTMX
|
|
||||||
mutation endpoint so each action gets a consistent re-render.
|
|
||||||
|
|
||||||
`form_overrides` lets a failed edit-save bounce show the user's
|
|
||||||
in-progress values instead of the row's stored values.
|
|
||||||
"""
|
|
||||||
db = get_database()
|
db = get_database()
|
||||||
with db.session() as sess:
|
with db.session() as sess:
|
||||||
source = sess.query(Source).filter(Source.id == source_id).first()
|
source = sess.query(Source).filter(Source.id == source_id).first()
|
||||||
if not source:
|
if not source:
|
||||||
raise HTTPException(status_code=404, detail="Source not found")
|
raise HTTPException(status_code=404, detail="Source not found")
|
||||||
source_data = _source_to_dict(source)
|
source_data = _source_to_dict(source)
|
||||||
extraction_data = (
|
extraction_data = _extraction_to_dict(source.extraction) if source.extraction else None
|
||||||
_extraction_to_dict(source.extraction) if source.extraction else None
|
|
||||||
)
|
|
||||||
|
|
||||||
if form_overrides:
|
|
||||||
source_data = {**source_data, **form_overrides}
|
|
||||||
|
|
||||||
return templates.TemplateResponse(
|
return templates.TemplateResponse(
|
||||||
request,
|
request,
|
||||||
@ -159,25 +137,10 @@ def _render_source_detail(
|
|||||||
{
|
{
|
||||||
"source": source_data,
|
"source": source_data,
|
||||||
"extraction": extraction_data,
|
"extraction": extraction_data,
|
||||||
"edit_mode": edit,
|
|
||||||
"flash": flash,
|
|
||||||
"error": error,
|
|
||||||
"domains": DOMAINS,
|
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@app.get("/sources/{source_id}", response_class=HTMLResponse)
|
|
||||||
async def source_detail(request: Request, source_id: int, edit: bool = False):
|
|
||||||
"""Source detail page with extraction results and accept/reject actions.
|
|
||||||
|
|
||||||
`?edit=1` opens the metadata editor inline. Editing exposes domain,
|
|
||||||
title, and focus — URL stays read-only since it's the UNIQUE dedupe
|
|
||||||
key for the queue and the embeddings identity tuple.
|
|
||||||
"""
|
|
||||||
return _render_source_detail(request, source_id, edit=edit)
|
|
||||||
|
|
||||||
|
|
||||||
_IN_FLIGHT_STATUSES = [
|
_IN_FLIGHT_STATUSES = [
|
||||||
SourceStatus.PENDING,
|
SourceStatus.PENDING,
|
||||||
SourceStatus.PULLED,
|
SourceStatus.PULLED,
|
||||||
@ -229,11 +192,17 @@ async def accept_source(request: Request, source_id: int):
|
|||||||
if not source:
|
if not source:
|
||||||
raise HTTPException(status_code=404, detail="Source not found")
|
raise HTTPException(status_code=404, detail="Source not found")
|
||||||
source.status = SourceStatus.ACCEPTED
|
source.status = SourceStatus.ACCEPTED
|
||||||
|
source_data = _source_to_dict(source)
|
||||||
|
extraction_data = _extraction_to_dict(source.extraction) if source.extraction else None
|
||||||
|
|
||||||
return _render_source_detail(
|
return templates.TemplateResponse(
|
||||||
request,
|
request,
|
||||||
source_id,
|
"source_detail.html",
|
||||||
flash="Accepted — will be included in next wiki compile.",
|
{
|
||||||
|
"source": source_data,
|
||||||
|
"extraction": extraction_data,
|
||||||
|
"flash": "Accepted — will be included in next wiki compile.",
|
||||||
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@ -246,80 +215,19 @@ async def reject_source(request: Request, source_id: int):
|
|||||||
if not source:
|
if not source:
|
||||||
raise HTTPException(status_code=404, detail="Source not found")
|
raise HTTPException(status_code=404, detail="Source not found")
|
||||||
source.status = SourceStatus.FAILED
|
source.status = SourceStatus.FAILED
|
||||||
|
source_data = _source_to_dict(source)
|
||||||
|
extraction_data = _extraction_to_dict(source.extraction) if source.extraction else None
|
||||||
|
|
||||||
return _render_source_detail(request, source_id, flash="Rejected.")
|
return templates.TemplateResponse(
|
||||||
|
|
||||||
|
|
||||||
@app.post("/sources/{source_id}/edit", response_class=HTMLResponse)
|
|
||||||
async def edit_source_metadata(
|
|
||||||
request: Request,
|
|
||||||
source_id: int,
|
|
||||||
domain: str = Form(...),
|
|
||||||
title: Optional[str] = Form(None),
|
|
||||||
focus: Optional[str] = Form(None),
|
|
||||||
):
|
|
||||||
"""HTMX edit-mode save. Updates domain/title/focus only — URL is
|
|
||||||
intentionally not editable (see service-layer comment)."""
|
|
||||||
db = get_database()
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
update_source_metadata(
|
|
||||||
sess,
|
|
||||||
source_id=source_id,
|
|
||||||
domain=domain,
|
|
||||||
title=title,
|
|
||||||
focus=focus,
|
|
||||||
)
|
|
||||||
except LookupError as exc:
|
|
||||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
|
||||||
except ValueError as exc:
|
|
||||||
# Bounce back to edit mode with the user's in-progress values so
|
|
||||||
# they don't have to retype them.
|
|
||||||
return _render_source_detail(
|
|
||||||
request,
|
request,
|
||||||
source_id,
|
"source_detail.html",
|
||||||
edit=True,
|
{
|
||||||
error=str(exc),
|
"source": source_data,
|
||||||
form_overrides={
|
"extraction": extraction_data,
|
||||||
"domain": domain,
|
"flash": "Rejected.",
|
||||||
"title": title or "",
|
|
||||||
"focus": focus or "",
|
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
return _render_source_detail(request, source_id, flash="Saved.")
|
|
||||||
|
|
||||||
|
|
||||||
@app.post("/sources/{source_id}/retry", response_class=HTMLResponse)
|
|
||||||
async def retry_source(
|
|
||||||
request: Request,
|
|
||||||
source_id: int,
|
|
||||||
mode: str = Form(RETRY_FULL),
|
|
||||||
):
|
|
||||||
"""HTMX retry/re-run. `mode=full` → PENDING (re-pull + re-process).
|
|
||||||
`mode=extract` → TRANSCRIBED (re-extract only; requires an existing
|
|
||||||
transcript). Always clears the claim pair and error_message."""
|
|
||||||
if mode not in RETRY_MODES:
|
|
||||||
raise HTTPException(status_code=400, detail=f"unknown mode {mode!r}")
|
|
||||||
|
|
||||||
db = get_database()
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
source = reset_source_for_retry(
|
|
||||||
sess, source_id=source_id, mode=mode
|
|
||||||
)
|
|
||||||
new_status = source.status.value
|
|
||||||
except LookupError as exc:
|
|
||||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
|
||||||
except ValueError as exc:
|
|
||||||
return _render_source_detail(request, source_id, error=str(exc))
|
|
||||||
|
|
||||||
if mode == RETRY_EXTRACT:
|
|
||||||
flash = f"Re-queued for extraction (status → {new_status})."
|
|
||||||
else:
|
|
||||||
flash = f"Re-queued from scratch (status → {new_status})."
|
|
||||||
return _render_source_detail(request, source_id, flash=flash)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# JSON API endpoints
|
# JSON API endpoints
|
||||||
@ -378,7 +286,6 @@ def _source_to_dict(s: Source) -> dict:
|
|||||||
"focus": s.focus,
|
"focus": s.focus,
|
||||||
"ingested_at": s.ingested_at.strftime("%Y-%m-%d %H:%M") if s.ingested_at else "",
|
"ingested_at": s.ingested_at.strftime("%Y-%m-%d %H:%M") if s.ingested_at else "",
|
||||||
"has_extraction": s.extraction is not None,
|
"has_extraction": s.extraction is not None,
|
||||||
"has_transcript": bool((s.transcript_text or "").strip()),
|
|
||||||
"error_message": s.error_message,
|
"error_message": s.error_message,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -11,74 +11,17 @@
|
|||||||
</svg>
|
</svg>
|
||||||
Back to sources
|
Back to sources
|
||||||
</a>
|
</a>
|
||||||
<div class="flex items-start justify-between gap-4">
|
<h2 class="text-2xl font-semibold text-gray-900">{{ source.title }}</h2>
|
||||||
<div class="min-w-0">
|
|
||||||
<h2 class="text-2xl font-semibold text-gray-900 break-words">{{ source.title }}</h2>
|
|
||||||
<a href="{{ source.url }}" target="_blank"
|
<a href="{{ source.url }}" target="_blank"
|
||||||
class="text-sm text-indigo-500 hover:text-indigo-700 break-all">{{ source.url }}</a>
|
class="text-sm text-indigo-500 hover:text-indigo-700 break-all">{{ source.url }}</a>
|
||||||
</div>
|
</div>
|
||||||
{% if not edit_mode %}
|
|
||||||
<a href="/sources/{{ source.id }}?edit=1"
|
|
||||||
class="shrink-0 px-3 py-1.5 text-sm border border-gray-300 text-gray-700 rounded hover:bg-gray-50 transition-colors">
|
|
||||||
Edit
|
|
||||||
</a>
|
|
||||||
{% endif %}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{% if flash %}
|
{% if flash %}
|
||||||
<div class="mb-4 px-4 py-3 bg-green-50 border border-green-200 text-green-800 rounded text-sm">
|
<div class="mb-4 px-4 py-3 bg-green-50 border border-green-200 text-green-800 rounded text-sm">
|
||||||
{{ flash }}
|
{{ flash }}
|
||||||
</div>
|
</div>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
{% if error %}
|
|
||||||
<div class="mb-4 px-4 py-3 bg-red-50 border border-red-200 text-red-800 rounded text-sm">
|
|
||||||
{{ error }}
|
|
||||||
</div>
|
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
{% if edit_mode %}
|
|
||||||
<!-- Edit mode: domain / title / focus. URL is intentionally read-only -->
|
|
||||||
<form hx-post="/sources/{{ source.id }}/edit"
|
|
||||||
hx-target="body"
|
|
||||||
hx-swap="innerHTML"
|
|
||||||
class="mb-8 bg-white rounded-lg border border-gray-200 p-5 space-y-4">
|
|
||||||
<div>
|
|
||||||
<label class="block text-xs font-semibold text-gray-500 uppercase tracking-wide mb-1">URL <span class="font-normal normal-case text-gray-400">(read-only — dedupe key)</span></label>
|
|
||||||
<input type="text" value="{{ source.url }}" disabled
|
|
||||||
class="w-full px-3 py-2 border border-gray-200 bg-gray-50 text-gray-500 text-sm rounded">
|
|
||||||
</div>
|
|
||||||
<div>
|
|
||||||
<label for="edit-title" class="block text-xs font-semibold text-gray-500 uppercase tracking-wide mb-1">Title</label>
|
|
||||||
<input type="text" id="edit-title" name="title" value="{{ source.title or '' }}"
|
|
||||||
class="w-full px-3 py-2 border border-gray-300 text-sm rounded focus:outline-none focus:ring-2 focus:ring-indigo-300">
|
|
||||||
</div>
|
|
||||||
<div>
|
|
||||||
<label for="edit-domain" class="block text-xs font-semibold text-gray-500 uppercase tracking-wide mb-1">Domain</label>
|
|
||||||
<select id="edit-domain" name="domain"
|
|
||||||
class="w-full px-3 py-2 border border-gray-300 text-sm rounded focus:outline-none focus:ring-2 focus:ring-indigo-300">
|
|
||||||
{% for d in domains %}
|
|
||||||
<option value="{{ d }}" {% if d == source.domain %}selected{% endif %}>{{ d }}</option>
|
|
||||||
{% endfor %}
|
|
||||||
</select>
|
|
||||||
</div>
|
|
||||||
<div>
|
|
||||||
<label for="edit-focus" class="block text-xs font-semibold text-gray-500 uppercase tracking-wide mb-1">Focus</label>
|
|
||||||
<textarea id="edit-focus" name="focus" rows="3"
|
|
||||||
class="w-full px-3 py-2 border border-gray-300 text-sm rounded focus:outline-none focus:ring-2 focus:ring-indigo-300">{{ source.focus or '' }}</textarea>
|
|
||||||
</div>
|
|
||||||
<div class="flex gap-2 pt-1">
|
|
||||||
<button type="submit"
|
|
||||||
class="px-4 py-2 bg-indigo-600 text-white text-sm font-semibold rounded hover:bg-indigo-700 transition-colors">
|
|
||||||
Save
|
|
||||||
</button>
|
|
||||||
<a href="/sources/{{ source.id }}"
|
|
||||||
class="px-4 py-2 border border-gray-300 text-gray-700 text-sm font-semibold rounded hover:bg-gray-50 transition-colors">
|
|
||||||
Cancel
|
|
||||||
</a>
|
|
||||||
</div>
|
|
||||||
</form>
|
|
||||||
{% else %}
|
|
||||||
<!-- Meta row -->
|
<!-- Meta row -->
|
||||||
<div class="flex flex-wrap gap-2 mb-6">
|
<div class="flex flex-wrap gap-2 mb-6">
|
||||||
{% set status_colors = {
|
{% set status_colors = {
|
||||||
@ -117,9 +60,9 @@
|
|||||||
</div>
|
</div>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
|
||||||
<!-- Actions row: accept/reject (when analyzed) + retry (always) -->
|
<!-- Accept / Reject actions (only when analyzed) -->
|
||||||
<div class="flex flex-wrap gap-3 mb-8">
|
|
||||||
{% if source.status == 'analyzed' %}
|
{% if source.status == 'analyzed' %}
|
||||||
|
<div class="flex gap-3 mb-8">
|
||||||
<button hx-post="/sources/{{ source.id }}/accept"
|
<button hx-post="/sources/{{ source.id }}/accept"
|
||||||
hx-target="body"
|
hx-target="body"
|
||||||
hx-swap="innerHTML"
|
hx-swap="innerHTML"
|
||||||
@ -132,31 +75,6 @@
|
|||||||
class="px-5 py-2 bg-red-500 text-white text-sm font-semibold rounded hover:bg-red-600 transition-colors">
|
class="px-5 py-2 bg-red-500 text-white text-sm font-semibold rounded hover:bg-red-600 transition-colors">
|
||||||
Reject
|
Reject
|
||||||
</button>
|
</button>
|
||||||
{% endif %}
|
|
||||||
|
|
||||||
<!-- Re-run: full reset → PENDING. Re-pulls and re-processes from scratch. -->
|
|
||||||
<button hx-post="/sources/{{ source.id }}/retry"
|
|
||||||
hx-vals='{"mode": "full"}'
|
|
||||||
hx-target="body"
|
|
||||||
hx-swap="innerHTML"
|
|
||||||
hx-confirm="Re-run this source from scratch? Status will reset to PENDING and the pipeline will re-pull{% if source.source_type == 'video' %} and re-transcribe (expensive){% endif %}."
|
|
||||||
class="px-5 py-2 bg-amber-500 text-white text-sm font-semibold rounded hover:bg-amber-600 transition-colors">
|
|
||||||
Re-run from scratch
|
|
||||||
</button>
|
|
||||||
|
|
||||||
{% if source.has_transcript %}
|
|
||||||
<!-- Light retry: skip the re-pull, just re-run LLM extraction. Works
|
|
||||||
for any source with stored body text (videos have transcripts,
|
|
||||||
articles have the trafilatura-extracted body). -->
|
|
||||||
<button hx-post="/sources/{{ source.id }}/retry"
|
|
||||||
hx-vals='{"mode": "extract"}'
|
|
||||||
hx-target="body"
|
|
||||||
hx-swap="innerHTML"
|
|
||||||
hx-confirm="Re-extract only — keep the existing transcript/body and re-run the LLM extraction?"
|
|
||||||
class="px-5 py-2 bg-amber-100 text-amber-800 text-sm font-semibold rounded hover:bg-amber-200 border border-amber-300 transition-colors">
|
|
||||||
Re-extract only
|
|
||||||
</button>
|
|
||||||
{% endif %}
|
|
||||||
</div>
|
</div>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
|
||||||
|
|||||||
@ -1,165 +0,0 @@
|
|||||||
"""Tests for the settings.local.toml overlay loader.
|
|
||||||
|
|
||||||
The overlay is the dev-side path for keeping host-specific secrets
|
|
||||||
(notably `[database].url`) out of the tracked `settings.toml` without
|
|
||||||
forcing every interactive shell to export env vars. The footgun this
|
|
||||||
test suite guards against: a one-key local override (e.g. a single
|
|
||||||
`[database] url = "..."`) silently wiping every other key in that
|
|
||||||
section. The merge is section-level, not shallow.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Pure-Python: _overlay_sections
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_overlay_preserves_sibling_keys_in_overlapping_section():
|
|
||||||
"""The exact regression target — local `url` must not wipe `pool_size`."""
|
|
||||||
from second_brain.config import _overlay_sections
|
|
||||||
|
|
||||||
base = {"database": {"url": "", "pool_size": 5}}
|
|
||||||
local = {"database": {"url": "postgresql://x"}}
|
|
||||||
|
|
||||||
out = _overlay_sections(base, local)
|
|
||||||
assert out["database"]["url"] == "postgresql://x"
|
|
||||||
assert out["database"]["pool_size"] == 5
|
|
||||||
|
|
||||||
|
|
||||||
def test_overlay_section_only_in_base_survives():
|
|
||||||
from second_brain.config import _overlay_sections
|
|
||||||
|
|
||||||
base = {"database": {"url": "a"}, "scheduler": {"interval": 30}}
|
|
||||||
local = {"database": {"url": "b"}}
|
|
||||||
|
|
||||||
out = _overlay_sections(base, local)
|
|
||||||
assert out["scheduler"] == {"interval": 30}
|
|
||||||
assert out["database"]["url"] == "b"
|
|
||||||
|
|
||||||
|
|
||||||
def test_overlay_section_only_in_local_appears():
|
|
||||||
from second_brain.config import _overlay_sections
|
|
||||||
|
|
||||||
base = {"database": {"url": "a"}}
|
|
||||||
local = {"embeddings": {"ollama_url": "http://x"}}
|
|
||||||
|
|
||||||
out = _overlay_sections(base, local)
|
|
||||||
assert out["database"] == {"url": "a"}
|
|
||||||
assert out["embeddings"] == {"ollama_url": "http://x"}
|
|
||||||
|
|
||||||
|
|
||||||
def test_overlay_scalar_in_overlay_replaces_dict_in_base():
|
|
||||||
"""Non-dict overlay values replace outright — no merge attempt."""
|
|
||||||
from second_brain.config import _overlay_sections
|
|
||||||
|
|
||||||
base = {"feature": {"a": 1, "b": 2}}
|
|
||||||
local = {"feature": "off"} # scalar wins outright
|
|
||||||
|
|
||||||
out = _overlay_sections(base, local)
|
|
||||||
assert out["feature"] == "off"
|
|
||||||
|
|
||||||
|
|
||||||
def test_overlay_dict_in_overlay_replaces_scalar_in_base():
|
|
||||||
from second_brain.config import _overlay_sections
|
|
||||||
|
|
||||||
base = {"feature": "off"}
|
|
||||||
local = {"feature": {"a": 1}}
|
|
||||||
|
|
||||||
out = _overlay_sections(base, local)
|
|
||||||
assert out["feature"] == {"a": 1}
|
|
||||||
|
|
||||||
|
|
||||||
def test_overlay_empty_overlay_is_identity():
|
|
||||||
from second_brain.config import _overlay_sections
|
|
||||||
|
|
||||||
base = {"database": {"url": "a", "pool_size": 5}}
|
|
||||||
assert _overlay_sections(base, {}) == base
|
|
||||||
assert _overlay_sections(base, None) == base
|
|
||||||
|
|
||||||
|
|
||||||
def test_overlay_only_merges_one_level():
|
|
||||||
"""Doc: the overlay is section-level. Nested dicts inside a section
|
|
||||||
replace, not deep-merge. This pins the documented depth."""
|
|
||||||
from second_brain.config import _overlay_sections
|
|
||||||
|
|
||||||
base = {"db": {"url": "a", "pool": {"min": 1, "max": 10}}}
|
|
||||||
local = {"db": {"pool": {"max": 20}}}
|
|
||||||
|
|
||||||
out = _overlay_sections(base, local)
|
|
||||||
# pool replaced wholesale — `min` does NOT survive. This is deliberate.
|
|
||||||
assert out["db"]["pool"] == {"max": 20}
|
|
||||||
assert out["db"]["url"] == "a"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# load_config: end-to-end with real TOML files on disk
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _write(path: Path, body: str) -> None:
|
|
||||||
path.write_text(body, encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _reset_config():
|
|
||||||
"""Drop the cached singleton before/after each test so overlay state
|
|
||||||
doesn't leak between cases."""
|
|
||||||
from second_brain.config import reset_config_singleton
|
|
||||||
|
|
||||||
reset_config_singleton()
|
|
||||||
yield
|
|
||||||
reset_config_singleton()
|
|
||||||
|
|
||||||
|
|
||||||
def test_load_config_picks_up_sibling_local_overlay(tmp_path):
|
|
||||||
from second_brain.config import load_config
|
|
||||||
|
|
||||||
base = tmp_path / "settings.toml"
|
|
||||||
local = tmp_path / "settings.local.toml"
|
|
||||||
_write(
|
|
||||||
base,
|
|
||||||
'[database]\nurl = ""\n\n[embeddings]\nollama_url = "http://ollama:11434"\nmodel = "nomic-embed-text"\n',
|
|
||||||
)
|
|
||||||
_write(
|
|
||||||
local,
|
|
||||||
'[database]\nurl = "postgresql+psycopg://lovebug:x@127.0.0.1:5433/petalbrain"\n\n[embeddings]\nollama_url = "http://127.0.0.1:11434"\n',
|
|
||||||
)
|
|
||||||
|
|
||||||
cfg = load_config(base)
|
|
||||||
# url from local wins; default model survives (not in local).
|
|
||||||
assert cfg.database["url"].startswith("postgresql+psycopg://")
|
|
||||||
assert cfg.embeddings["ollama_url"] == "http://127.0.0.1:11434"
|
|
||||||
assert cfg.embeddings["model"] == "nomic-embed-text"
|
|
||||||
|
|
||||||
|
|
||||||
def test_load_config_no_local_overlay_is_inert(tmp_path):
|
|
||||||
from second_brain.config import load_config
|
|
||||||
|
|
||||||
base = tmp_path / "settings.toml"
|
|
||||||
_write(base, '[database]\nurl = "from-base"\n')
|
|
||||||
|
|
||||||
cfg = load_config(base)
|
|
||||||
assert cfg.database["url"] == "from-base"
|
|
||||||
|
|
||||||
|
|
||||||
def test_load_config_local_overlay_at_explicit_path(tmp_path):
|
|
||||||
"""Doc: when an explicit (or SECOND_BRAIN_CONFIG) path is used, the
|
|
||||||
overlay sibling is resolved next to THAT path — not next to the
|
|
||||||
repo's config dir."""
|
|
||||||
from second_brain.config import load_config
|
|
||||||
|
|
||||||
subdir = tmp_path / "etc"
|
|
||||||
subdir.mkdir()
|
|
||||||
base = subdir / "settings.toml"
|
|
||||||
local = subdir / "settings.local.toml"
|
|
||||||
_write(base, '[database]\nurl = "base"\n')
|
|
||||||
_write(local, '[database]\nurl = "overlay-wins"\n')
|
|
||||||
|
|
||||||
cfg = load_config(base)
|
|
||||||
assert cfg.database["url"] == "overlay-wins"
|
|
||||||
@ -1,258 +0,0 @@
|
|||||||
"""Tests for the source edit-metadata + retry-reset service helpers.
|
|
||||||
|
|
||||||
Live-DB pattern matches `tests/test_claim.py` / `tests/test_playlist.py`:
|
|
||||||
both groups auto-skip when no petalbrain connection is configured. The
|
|
||||||
service layer is where edit/retry logic lives (the web routes + CLI are
|
|
||||||
thin wrappers around it), so this is where the meaningful coverage is.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import uuid
|
|
||||||
|
|
||||||
import psycopg
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
def _db_reachable() -> bool:
|
|
||||||
url = os.environ.get("SECOND_BRAIN_DATABASE_URL") or os.environ.get(
|
|
||||||
"HERBYLAB_DATABASE_URL"
|
|
||||||
)
|
|
||||||
if not url:
|
|
||||||
return False
|
|
||||||
raw = url.replace("postgresql+psycopg://", "postgresql://", 1)
|
|
||||||
try:
|
|
||||||
with psycopg.connect(raw, connect_timeout=2):
|
|
||||||
return True
|
|
||||||
except Exception:
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.skipif(
|
|
||||||
not _db_reachable(),
|
|
||||||
reason="needs SECOND_BRAIN_DATABASE_URL pointed at a real petalbrain",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _seed_source(db, **overrides):
|
|
||||||
"""Insert a Source row and return (id, url). Caller cleans up by id."""
|
|
||||||
from second_brain.models import Source, SourceStatus, SourceType, utcnow
|
|
||||||
|
|
||||||
tag = uuid.uuid4().hex[:8]
|
|
||||||
defaults = dict(
|
|
||||||
url=f"https://example.test/edit-retry/{tag}",
|
|
||||||
title=f"seed-{tag}",
|
|
||||||
domain="development",
|
|
||||||
source_type=SourceType.ARTICLE,
|
|
||||||
status=SourceStatus.PENDING,
|
|
||||||
ingested_at=utcnow(),
|
|
||||||
)
|
|
||||||
defaults.update(overrides)
|
|
||||||
with db.session() as sess:
|
|
||||||
src = Source(**defaults)
|
|
||||||
sess.add(src)
|
|
||||||
sess.flush()
|
|
||||||
return src.id, src.url
|
|
||||||
|
|
||||||
|
|
||||||
def _cleanup(db, source_id):
|
|
||||||
from second_brain.models import Source
|
|
||||||
|
|
||||||
with db.session() as sess:
|
|
||||||
sess.query(Source).filter(Source.id == source_id).delete(
|
|
||||||
synchronize_session=False
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# update_source_metadata
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_update_metadata_writes_domain_title_focus():
|
|
||||||
from second_brain.database import Database, reset_database_singleton
|
|
||||||
from second_brain.models import Source
|
|
||||||
from second_brain.sources_service import update_source_metadata
|
|
||||||
|
|
||||||
reset_database_singleton()
|
|
||||||
db = Database()
|
|
||||||
sid, _ = _seed_source(db)
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
update_source_metadata(
|
|
||||||
sess,
|
|
||||||
source_id=sid,
|
|
||||||
domain="homelab",
|
|
||||||
title="new title",
|
|
||||||
focus="new focus directive",
|
|
||||||
)
|
|
||||||
with db.session() as sess:
|
|
||||||
row = sess.query(Source).filter(Source.id == sid).first()
|
|
||||||
assert row.domain == "homelab"
|
|
||||||
assert row.title == "new title"
|
|
||||||
assert row.focus == "new focus directive"
|
|
||||||
assert row.updated_at is not None
|
|
||||||
finally:
|
|
||||||
_cleanup(db, sid)
|
|
||||||
reset_database_singleton()
|
|
||||||
|
|
||||||
|
|
||||||
def test_update_metadata_blank_strings_become_null():
|
|
||||||
from second_brain.database import Database, reset_database_singleton
|
|
||||||
from second_brain.models import Source
|
|
||||||
from second_brain.sources_service import update_source_metadata
|
|
||||||
|
|
||||||
reset_database_singleton()
|
|
||||||
db = Database()
|
|
||||||
sid, _ = _seed_source(db)
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
update_source_metadata(
|
|
||||||
sess, source_id=sid, domain="development", title=" ", focus=""
|
|
||||||
)
|
|
||||||
with db.session() as sess:
|
|
||||||
row = sess.query(Source).filter(Source.id == sid).first()
|
|
||||||
assert row.title is None
|
|
||||||
assert row.focus is None
|
|
||||||
finally:
|
|
||||||
_cleanup(db, sid)
|
|
||||||
reset_database_singleton()
|
|
||||||
|
|
||||||
|
|
||||||
def test_update_metadata_rejects_unknown_domain():
|
|
||||||
from second_brain.database import Database, reset_database_singleton
|
|
||||||
from second_brain.sources_service import update_source_metadata
|
|
||||||
|
|
||||||
reset_database_singleton()
|
|
||||||
db = Database()
|
|
||||||
sid, _ = _seed_source(db)
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
with pytest.raises(ValueError, match="unknown domain"):
|
|
||||||
update_source_metadata(sess, source_id=sid, domain="not-a-domain")
|
|
||||||
finally:
|
|
||||||
_cleanup(db, sid)
|
|
||||||
reset_database_singleton()
|
|
||||||
|
|
||||||
|
|
||||||
def test_update_metadata_missing_source_raises_lookup():
|
|
||||||
from second_brain.database import Database, reset_database_singleton
|
|
||||||
from second_brain.sources_service import update_source_metadata
|
|
||||||
|
|
||||||
reset_database_singleton()
|
|
||||||
db = Database()
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
with pytest.raises(LookupError):
|
|
||||||
update_source_metadata(
|
|
||||||
sess, source_id=-99999, domain="development"
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
reset_database_singleton()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# reset_source_for_retry
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_retry_full_resets_to_pending_and_clears_claim():
|
|
||||||
from second_brain.database import Database, reset_database_singleton
|
|
||||||
from second_brain.models import Source, SourceStatus, utcnow
|
|
||||||
from second_brain.sources_service import reset_source_for_retry
|
|
||||||
|
|
||||||
reset_database_singleton()
|
|
||||||
db = Database()
|
|
||||||
sid, _ = _seed_source(
|
|
||||||
db,
|
|
||||||
status=SourceStatus.FAILED,
|
|
||||||
error_message="boom",
|
|
||||||
claimed_by="worker-x",
|
|
||||||
claimed_at=utcnow(),
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
reset_source_for_retry(sess, source_id=sid, mode="full")
|
|
||||||
with db.session() as sess:
|
|
||||||
row = sess.query(Source).filter(Source.id == sid).first()
|
|
||||||
assert row.status == SourceStatus.PENDING
|
|
||||||
assert row.claimed_by is None
|
|
||||||
assert row.claimed_at is None
|
|
||||||
assert row.error_message is None
|
|
||||||
finally:
|
|
||||||
_cleanup(db, sid)
|
|
||||||
reset_database_singleton()
|
|
||||||
|
|
||||||
|
|
||||||
def test_retry_extract_sets_transcribed_when_transcript_present():
|
|
||||||
from second_brain.database import Database, reset_database_singleton
|
|
||||||
from second_brain.models import Source, SourceStatus
|
|
||||||
from second_brain.sources_service import reset_source_for_retry
|
|
||||||
|
|
||||||
reset_database_singleton()
|
|
||||||
db = Database()
|
|
||||||
sid, _ = _seed_source(
|
|
||||||
db,
|
|
||||||
status=SourceStatus.ANALYZED,
|
|
||||||
transcript_text="word " * 50,
|
|
||||||
error_message="prior failure",
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
reset_source_for_retry(sess, source_id=sid, mode="extract")
|
|
||||||
with db.session() as sess:
|
|
||||||
row = sess.query(Source).filter(Source.id == sid).first()
|
|
||||||
assert row.status == SourceStatus.TRANSCRIBED
|
|
||||||
assert row.error_message is None
|
|
||||||
assert row.transcript_text # transcript preserved
|
|
||||||
finally:
|
|
||||||
_cleanup(db, sid)
|
|
||||||
reset_database_singleton()
|
|
||||||
|
|
||||||
|
|
||||||
def test_retry_extract_without_transcript_raises():
|
|
||||||
from second_brain.database import Database, reset_database_singleton
|
|
||||||
from second_brain.models import SourceStatus
|
|
||||||
from second_brain.sources_service import reset_source_for_retry
|
|
||||||
|
|
||||||
reset_database_singleton()
|
|
||||||
db = Database()
|
|
||||||
sid, _ = _seed_source(db, status=SourceStatus.PENDING, transcript_text=None)
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
with pytest.raises(ValueError, match="requires an existing transcript"):
|
|
||||||
reset_source_for_retry(sess, source_id=sid, mode="extract")
|
|
||||||
finally:
|
|
||||||
_cleanup(db, sid)
|
|
||||||
reset_database_singleton()
|
|
||||||
|
|
||||||
|
|
||||||
def test_retry_unknown_mode_raises():
|
|
||||||
from second_brain.database import Database, reset_database_singleton
|
|
||||||
from second_brain.sources_service import reset_source_for_retry
|
|
||||||
|
|
||||||
reset_database_singleton()
|
|
||||||
db = Database()
|
|
||||||
sid, _ = _seed_source(db)
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
with pytest.raises(ValueError, match="unknown retry mode"):
|
|
||||||
reset_source_for_retry(sess, source_id=sid, mode="bogus")
|
|
||||||
finally:
|
|
||||||
_cleanup(db, sid)
|
|
||||||
reset_database_singleton()
|
|
||||||
|
|
||||||
|
|
||||||
def test_retry_missing_source_raises_lookup():
|
|
||||||
from second_brain.database import Database, reset_database_singleton
|
|
||||||
from second_brain.sources_service import reset_source_for_retry
|
|
||||||
|
|
||||||
reset_database_singleton()
|
|
||||||
db = Database()
|
|
||||||
try:
|
|
||||||
with db.session() as sess:
|
|
||||||
with pytest.raises(LookupError):
|
|
||||||
reset_source_for_retry(sess, source_id=-99999, mode="full")
|
|
||||||
finally:
|
|
||||||
reset_database_singleton()
|
|
||||||
Loading…
Reference in New Issue
Block a user