Travis Herbranson c4793f3a7f switch extractor to Claude CLI backend + pipeline fixes

Run extraction under the Max OAuth subscription via `claude -p` instead
of the per-token Anthropic API. The new src/second_brain/llm/claude_cli.py
spawns the CLI in a hermetic tempdir so the host project's CLAUDE.md,
hooks, MCP config, and settings don't leak into the prompt. Uses
--json-schema with LLMExtraction.model_json_schema() so the CLI guarantees
valid structured output — replaces the brittle markdown-fence stripping
in the old engine. The Anthropic SDK is preserved as an optional "api"
backend selectable via config.

While in here, fix a handful of blockers that the smoke test surfaced:
- scheduler filtered ANALYZED instead of TRANSCRIBED, so it never
  actually advanced any sources
- process command read sources in a closed session, raising
  DetachedInstanceError before any work happened
- config.prompts_dir walked one parent too many, resolving outside
  the project and forcing the fallback prompt for every domain
- compiler called git rev-parse against a vault that was never
  git-init'd; now auto-inits with an empty seed commit and skips empty
  commits cleanly
- datetime.utcnow() deprecated in 3.12+ — single utcnow() helper in
  models.py keeps naive UTC semantics so no DB migration is needed
- sess.query(...).get() deprecated in SA 2.x → sess.get(...)
- dead `import anthropic` removed from compiler

Smoke test (article → process → accept → compile) succeeds end-to-end
with ANTHROPIC_API_KEY unset. a-review run saved under reviews/.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

2026-05-24 21:09:20 -04:00

3.7 KiB

Raw Blame History

a-review — 2026-05-24

Model: gemini Diff base: HEAD (9 files changed, 364 insertions(+), 152 deletions(-)) Session reference: none Context: CLAUDE.md + session notes + expanded diff

Here is a review of the code changes.

[WARNING] Potential for brittle path resolution

File: src/second_brain/config.py
Location: Line 94, prompts_dir property
Issue: The prompts_dir property calculates the project's root directory using Path(__file__).resolve().parent.parent.parent. This kind of relative path traversal (../../..) is brittle and can easily break if config.py or the directory structure is refactored.
Suggestion: For more complex projects, libraries like importlib.resources (for Python 3.9+) are a more robust way to locate package data. For the current project size, this is a minor issue, but it's worth being aware of the fragility.

[WARNING] Potential race condition in `process` command

File: src/second_brain/main.py
Location: Line 150 (process command)
Issue: The command first queries for a list of all eligible source_ids, and then iterates through this list, processing each one. If two instances of second-brain process were run concurrently, they would both get the same list of IDs and attempt to process the same sources, leading to duplicated work and potentially errors or race conditions upon writing results to the database.
Suggestion: This is a low-risk issue for a CLI tool that is likely run as a singleton. However, for a more robust system, processing should claim an item atomically. This could be done by querying for a single processable item within the loop, or by using a database-level lock (SELECT ... FOR UPDATE), to ensure a given source is only picked up by one process.

[INFO] Bug fix in scheduler logic

File: src/second_brain/scheduler/runner.py
Location: Line 122
Issue: The comments and code indicate a critical bug was fixed. The scheduler was previously attempting to process sources that were already in the ANALYZED state, causing it to do no useful work.
Fix: The query has been corrected to target sources in the TRANSCRIBED state (.filter(Source.status == SourceStatus.TRANSCRIBED)), which aligns with the documented pipeline flow.

[INFO] Robustness improvements in wiki compiler

File: src/second_brain/compiler/wiki.py
Location: _ensure_git_repo and _git_commit methods
Issue: The git integration is now significantly more robust.
Fix: The compiler now automatically initializes a git repository in the vault (_ensure_git_repo) if one doesn't exist, preventing errors on a fresh setup. Furthermore, the _git_commit logic now checks for staged changes before attempting a commit, preventing the creation of empty commits. These are high-quality, thoughtful improvements.

Summary of Changes

Overall, these are high-quality changes that introduce a significant and well-implemented architectural improvement (the CLI-based extractor backend) while also fixing several bugs and improving system robustness.

The new CLI backend is a clever approach that leverages the claude tool's schema enforcement to improve the reliability of JSON extraction, while also making the anthropic SDK an optional dependency.

The bug fix in the scheduler is critical for the system's core function. The improvements to the wiki compiler's git handling show good attention to detail and user experience. The minor warnings are about potential future issues rather than immediate bugs and do not detract from the overall quality of the submission. The code aligns well with the project's stated architecture and demonstrates a clear, positive evolution of the system.

3.7 KiB Raw Blame History