--- created: '2026-05-15' path: Sources/Dev project: zero-check-refactor status: active tags: - python - go - automation - claude-code type: project-plan updated: '2026-05-15' --- ## Goal Refactor the `zero-check` agentic skill to eliminate silent-fail modes and reduce bash complexity. Move source-file discovery from custom bash `find` invocations to the native discovery engines of each underlying tool (pytest, ruff, semgrep). Add an "inconclusive" guard so an all-skipped run no longer reports green. Update the global Python project scaffold to carry the project-side boundary rules that the gauntlet stops enforcing. ## Locked Decisions - **Stay on bash.** The orchestrator is dumb glue calling other CLIs — exactly the job bash is built for. No rewrite to Python/Go/just/pre-commit at this stage; revisit only if the gauntlet grows into non-deterministic agent-output validation. - **Manifest-driven routing.** Bash detects ecosystem via `pyproject.toml`, `requirements*.txt`, `uv.lock`, `go.mod`. Bash does not enumerate source files. - **Native discovery.** ruff, semgrep, pytest, go test all walk their own trees and respect project-level exclude config. - **Pytest exit code 5 → skipped (exit 2).** "No tests collected" is an honest skip, not a vacuous pass. - **Inconclusive guard.** A run where every check skipped is `overall: failed` with `inconclusive: true` in `results.json`. - **Boundary rules live in the project.** Non-standard venv names (`env/`, `app_env/`) and other project-specific exclusions belong in `pyproject.toml [tool.pytest.ini_options] norecursedirs`, not in the gauntlet. - **Three-state exit code contract preserved.** `0 = passed`, `1 = failed`, `2 = skipped`. Sub-scripts continue to honor this; orchestrator continues to route on it. ## Open Items - [ ] Decide whether to add a JSON-schema validator for `results.json` (defensive, but probably overkill at current scope) - [ ] Decide whether to add a `--strict` flag to `run-all.sh` that fails on any skip, not just all-skip (probably not — defeats the point of skipped-as-a-state) - [ ] Smoke-test the refactor against at least one Python project, one Go project, and one polyglot/mixed project before adopting as default ## Phases ### Phase 1 — Drop in the refactored scripts Replace the six scripts in `~/zero-check-pipeline/validate/` with the refactored versions drafted in chat. Files: - `run-all.sh` — adds inconclusive guard, adds `inconclusive` field to `results.json` - `secrets.sh` — unchanged (gitleaks already does its own tree walk) - `lint.sh` — manifest-routed, source-file probe removed - `sast.sh` — manifest-routed, source-file probe removed - `deps.sh` — minor normalization (find pattern → `-print -quit` for consistency) - `tests.sh` — manifest-routed, pytest exit 5 mapped to skipped, no source-file probe Tasks: - [ ] Back up current scripts: `cp -r ~/zero-check-pipeline/validate ~/zero-check-pipeline/validate.bak` - [ ] Drop in the six new scripts from the chat artifacts - [ ] `chmod +x ~/zero-check-pipeline/validate/*.sh` - [ ] `bash -n` syntax-check each (already verified clean in draft) ### Phase 2 — Smoke tests Validate the refactor against real projects before declaring it the default: - [ ] Run against herbylab MCP server (Python + uv): expect lint/sast/deps/tests all pass, secrets clean - [ ] Run against a Go project (pick one from gitea): expect Go branches exercise - [ ] Run against an empty scratch dir: expect `inconclusive: true`, overall failed - [ ] Run against a Python project with no tests yet: expect `tests` → skipped, not green - [ ] Confirm `results.json` shape is unchanged for downstream consumers (just gains the `inconclusive` field) ### Phase 3 — Global Python scaffold update Move the project-side boundary rules into the default `pyproject.toml` template so every new Python project carries them. - [ ] Locate the current Python scaffold (uv init template / cookiecutter / personal snippets — wherever new projects start from) - [ ] Add this block to the canonical scaffold: ```toml [tool.pytest.ini_options] norecursedirs = [ "env", "app_env", ".venv", "venv", "node_modules", "build", "dist", ".git", ] ``` - [ ] Document the rationale inline (one-line comment: "Excludes for tools that walk the tree; keep gauntlet agnostic") - [ ] If multiple scaffolds exist (e.g., one for FastMCP servers, one for plain libraries), update all of them - [ ] Backfill the snippet into existing active Python projects on the tower as opportunity arises — not a forced migration ### Phase 4 — SKILL.md hygiene The skill file at `~/.claude/skills/zero-check/SKILL.md` does not need to change — the contract (run-all.sh → results.json → retry loop) is preserved. But it's worth a once-over: - [ ] Verify the retry-loop guidance still makes sense with the inconclusive state (an inconclusive failure should not trigger a fix loop — there's nothing to fix; it should escalate to the human) - [ ] Consider adding: "If `inconclusive: true` in results.json, STOP and ask the user — do not retry." Inconclusive means the gauntlet did not validate; retrying won't change that. ## Notes ### What changed and why The original gauntlet had a class of silent failures rooted in bash trying to mimic the discovery engines of the tools it orchestrates. Two specific failure modes: 1. **SIGPIPE under `pipefail`.** `find ... | head -1` on dense trees could die with exit 141, killing the script before any output. The `-print -quit` workaround was already in place but only on some scripts. 2. **Vacuous passes.** A project with no `.py` files exited 0 from lint/sast/tests, contributing to a green overall result even though nothing was checked. The refactor addresses both by removing the bash-side file walking entirely (mode 1 cannot occur on a manifest lookup) and by adding the inconclusive guard (mode 2 is now flagged). ### Why not rewrite in Python Considered and rejected for now. The gauntlet's current job is shelling out to other CLI tools and aggregating exit codes — bash is genuinely correct for this. A Python rewrite would buy real value only if the gauntlet evolves into something more sophisticated (e.g., LLM-driven semantic validation of agent output, per the GitHub article on agentic behavior validation). Revisit then. ### Reference: refactored scripts The six refactored scripts were drafted in conversation and produced as downloadable artifacts. Drop-in replacements for the files in `~/zero-check-pipeline/validate/`. The diff against the originals is: - All five sub-scripts: source-file `find` calls replaced with manifest detection - `tests.sh`: pytest exit code 5 explicitly mapped to skipped state - `lint.sh`, `tests.sh` (Go branch): `cd $gomod_dir && ./...` pattern adopted - `run-all.sh`: inconclusive guard added after the check loop; `inconclusive` field added to `results.json` schema ### Reference: GitHub article (parking lot) Pending review: . Relevant to the question of whether the gauntlet should grow beyond deterministic checks. Decision deferred to a separate session.