From e43eacc5afc506b3477004d6df8babcc483d3864 Mon Sep 17 00:00:00 2001 From: Lovebug MCP Date: Fri, 15 May 2026 11:42:30 +0000 Subject: [PATCH] =?UTF-8?q?mcp:=20project-plan=20=E2=80=94=20zero-check-re?= =?UTF-8?q?factor?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- Sources/Dev/zero-check-refactor.md | 122 +++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 Sources/Dev/zero-check-refactor.md diff --git a/Sources/Dev/zero-check-refactor.md b/Sources/Dev/zero-check-refactor.md new file mode 100644 index 0000000..4fc2fad --- /dev/null +++ b/Sources/Dev/zero-check-refactor.md @@ -0,0 +1,122 @@ +--- +created: '2026-05-15' +path: Sources/Dev +project: zero-check-refactor +status: active +tags: +- python +- go +- automation +- claude-code +type: project-plan +updated: '2026-05-15' +--- + +## Goal + +Refactor the `zero-check` agentic skill to eliminate silent-fail modes and reduce bash complexity. Move source-file discovery from custom bash `find` invocations to the native discovery engines of each underlying tool (pytest, ruff, semgrep). Add an "inconclusive" guard so an all-skipped run no longer reports green. Update the global Python project scaffold to carry the project-side boundary rules that the gauntlet stops enforcing. + +## Locked Decisions + +- **Stay on bash.** The orchestrator is dumb glue calling other CLIs — exactly the job bash is built for. No rewrite to Python/Go/just/pre-commit at this stage; revisit only if the gauntlet grows into non-deterministic agent-output validation. +- **Manifest-driven routing.** Bash detects ecosystem via `pyproject.toml`, `requirements*.txt`, `uv.lock`, `go.mod`. Bash does not enumerate source files. +- **Native discovery.** ruff, semgrep, pytest, go test all walk their own trees and respect project-level exclude config. +- **Pytest exit code 5 → skipped (exit 2).** "No tests collected" is an honest skip, not a vacuous pass. +- **Inconclusive guard.** A run where every check skipped is `overall: failed` with `inconclusive: true` in `results.json`. +- **Boundary rules live in the project.** Non-standard venv names (`env/`, `app_env/`) and other project-specific exclusions belong in `pyproject.toml [tool.pytest.ini_options] norecursedirs`, not in the gauntlet. +- **Three-state exit code contract preserved.** `0 = passed`, `1 = failed`, `2 = skipped`. Sub-scripts continue to honor this; orchestrator continues to route on it. + +## Open Items + +- [ ] Decide whether to add a JSON-schema validator for `results.json` (defensive, but probably overkill at current scope) +- [ ] Decide whether to add a `--strict` flag to `run-all.sh` that fails on any skip, not just all-skip (probably not — defeats the point of skipped-as-a-state) +- [ ] Smoke-test the refactor against at least one Python project, one Go project, and one polyglot/mixed project before adopting as default + +## Phases + +### Phase 1 — Drop in the refactored scripts + +Replace the six scripts in `~/zero-check-pipeline/validate/` with the refactored versions drafted in chat. Files: + +- `run-all.sh` — adds inconclusive guard, adds `inconclusive` field to `results.json` +- `secrets.sh` — unchanged (gitleaks already does its own tree walk) +- `lint.sh` — manifest-routed, source-file probe removed +- `sast.sh` — manifest-routed, source-file probe removed +- `deps.sh` — minor normalization (find pattern → `-print -quit` for consistency) +- `tests.sh` — manifest-routed, pytest exit 5 mapped to skipped, no source-file probe + +Tasks: +- [ ] Back up current scripts: `cp -r ~/zero-check-pipeline/validate ~/zero-check-pipeline/validate.bak` +- [ ] Drop in the six new scripts from the chat artifacts +- [ ] `chmod +x ~/zero-check-pipeline/validate/*.sh` +- [ ] `bash -n` syntax-check each (already verified clean in draft) + +### Phase 2 — Smoke tests + +Validate the refactor against real projects before declaring it the default: + +- [ ] Run against herbylab MCP server (Python + uv): expect lint/sast/deps/tests all pass, secrets clean +- [ ] Run against a Go project (pick one from gitea): expect Go branches exercise +- [ ] Run against an empty scratch dir: expect `inconclusive: true`, overall failed +- [ ] Run against a Python project with no tests yet: expect `tests` → skipped, not green +- [ ] Confirm `results.json` shape is unchanged for downstream consumers (just gains the `inconclusive` field) + +### Phase 3 — Global Python scaffold update + +Move the project-side boundary rules into the default `pyproject.toml` template so every new Python project carries them. + +- [ ] Locate the current Python scaffold (uv init template / cookiecutter / personal snippets — wherever new projects start from) +- [ ] Add this block to the canonical scaffold: + + ```toml + [tool.pytest.ini_options] + norecursedirs = [ + "env", + "app_env", + ".venv", + "venv", + "node_modules", + "build", + "dist", + ".git", + ] + ``` + +- [ ] Document the rationale inline (one-line comment: "Excludes for tools that walk the tree; keep gauntlet agnostic") +- [ ] If multiple scaffolds exist (e.g., one for FastMCP servers, one for plain libraries), update all of them +- [ ] Backfill the snippet into existing active Python projects on the tower as opportunity arises — not a forced migration + +### Phase 4 — SKILL.md hygiene + +The skill file at `~/.claude/skills/zero-check/SKILL.md` does not need to change — the contract (run-all.sh → results.json → retry loop) is preserved. But it's worth a once-over: + +- [ ] Verify the retry-loop guidance still makes sense with the inconclusive state (an inconclusive failure should not trigger a fix loop — there's nothing to fix; it should escalate to the human) +- [ ] Consider adding: "If `inconclusive: true` in results.json, STOP and ask the user — do not retry." Inconclusive means the gauntlet did not validate; retrying won't change that. + +## Notes + +### What changed and why + +The original gauntlet had a class of silent failures rooted in bash trying to mimic the discovery engines of the tools it orchestrates. Two specific failure modes: + +1. **SIGPIPE under `pipefail`.** `find ... | head -1` on dense trees could die with exit 141, killing the script before any output. The `-print -quit` workaround was already in place but only on some scripts. +2. **Vacuous passes.** A project with no `.py` files exited 0 from lint/sast/tests, contributing to a green overall result even though nothing was checked. + +The refactor addresses both by removing the bash-side file walking entirely (mode 1 cannot occur on a manifest lookup) and by adding the inconclusive guard (mode 2 is now flagged). + +### Why not rewrite in Python + +Considered and rejected for now. The gauntlet's current job is shelling out to other CLI tools and aggregating exit codes — bash is genuinely correct for this. A Python rewrite would buy real value only if the gauntlet evolves into something more sophisticated (e.g., LLM-driven semantic validation of agent output, per the GitHub article on agentic behavior validation). Revisit then. + +### Reference: refactored scripts + +The six refactored scripts were drafted in conversation and produced as downloadable artifacts. Drop-in replacements for the files in `~/zero-check-pipeline/validate/`. The diff against the originals is: + +- All five sub-scripts: source-file `find` calls replaced with manifest detection +- `tests.sh`: pytest exit code 5 explicitly mapped to skipped state +- `lint.sh`, `tests.sh` (Go branch): `cd $gomod_dir && ./...` pattern adopted +- `run-all.sh`: inconclusive guard added after the check loop; `inconclusive` field added to `results.json` schema + +### Reference: GitHub article (parking lot) + +Pending review: . Relevant to the question of whether the gauntlet should grow beyond deterministic checks. Decision deferred to a separate session. \ No newline at end of file