pbs-projects/Tech/Projects/zero-check-pipeline.md

project	type	status	path	tags	created	updated
zero-check-pipeline	project-plan	active	Tech/Projects	homelab devops automation claude-code	2026-04-18	2026-04-18

Zero-Check Validation Pipeline

A reusable, language-aware validation pipeline for post-LLM code generation. Treats all generated code as untrusted and requires it to prove safety before promotion.

Core Principles

• LLM-agnostic — works on code from Claude, Gemini, or any generator
• Language-aware — detects Python and Go, runs appropriate tool chain
• Read-only checks — the gauntlet scans and reports, never modifies
• Scripts are locked — the LLM can fix code but cannot weaken the checks
• Phased commits — commit at phase boundaries for rollback granularity
• Git is the notebook, the gauntlet is the quality gate

Pipeline Flow

Phase 1 — Code Generation

• LLM generates code on Ubuntu server (us-test-authy)
• Commit at phase boundaries as work progresses

Phase 2 — Gauntlet Loop (automated, max 3 iterations)

• Secret detection (gitleaks)
• Linting (ruff for Python, golangci-lint for Go)
• SAST (semgrep with OWASP + language rulesets)
• Dependency audit (pip-audit for Python, govulncheck for Go)
• Unit tests (pytest for Python, go test for Go)
• Output is structured JSON so the LLM can parse failures and fix
• Each iteration gets a commit for traceability
• If not green after 3 attempts, escalate to human

Phase 3 — Container Verification (one-shot, not in the loop)

• Build from Dockerfile in clean environment
• Run full test suite inside container
• Docker Compose for Tier 2 integration tests (databases, Redis, etc.)
• Failure here gets kicked back to LLM as a specific task

Phase 4 — Architect Sweep (human, ~5 minutes)

• Review git diff from last clean baseline
• Check for outbound calls (requests, httpx, urllib)
• Check for auth bypasses (try/except: pass, commented-out decorators)
• Check for obfuscation (unexplained Base64/Hex strings)
• Approve or reject

Testing Tiers

Tier 1 — Self-contained (automated loop handles this)

• Linting, SAST, secrets, dependency audit, unit tests with mocks
• No network, no services, no auth required

Tier 2 — Local services (container verification handles this)

• App + database/Redis/queue via Docker Compose
• Isolated network, disposable, no real credentials

Tier 3 — External services (manual, deferred)

• Real OAuth, external APIs, Cloudflare integration
• Mock boundaries in automated tests, manual smoke test during architect sweep
• Dedicated test environment is a future evolution

Validation Artifacts

validate/results.json — Machine-readable pipeline state. Timestamp, pass/fail per check, iteration count, current status. Any new Claude session reads this to resume context.

validate/SUMMARY.md — Human-readable validation summary. Lives in repo, visible in Gitea. Browsable from any device.

Both committed and pushed with the project code.

Project Structure

validate/
  secrets.sh        # runs gitleaks
  lint.sh           # runs ruff or golangci-lint
  sast.sh           # runs semgrep
  deps.sh           # runs pip-audit or govulncheck
  tests.sh          # runs pytest or go test
  run-all.sh        # calls each in order, stops on failure
  results.json      # machine-readable output (generated)
  SUMMARY.md        # human-readable output (generated)

Each script is 10-20 lines, single purpose, human-readable.

Orchestration

• Claude Code is the meta-controller (the brain)
• Skills define the playbook (what to run, in what order, retry rules)
• Scripts are the hands (each runs one tool, returns structured output)
• No separate app needed — Claude Code + skills + scripts covers it

Build Order

• Individual validation scripts (secrets, lint, sast, deps, tests)
• run-all.sh orchestrator with language detection
• Structured JSON output (results.json)
• Human-readable summary generation (SUMMARY.md)
• Claude Code skill for automated gauntlet loop with retry logic
• Dockerfile template for container verification
• Architect sweep checklist document
• Docker Compose template for Tier 2 integration tests

Evolution Path

• Current (v1): Shell scripts + Claude Code skill, run locally
• v2: Taskfile (YAML-based task runner) if orchestration logic gets complex
• v3: GitHub Actions as safety net on push
• Future: Dashboard app for cross-project validation history (dog-food candidate for this pipeline)

Deferred

• GitHub Actions integration
• Tier 3 external integration test environment
• Reporting/trend dashboards
• act (local GitHub Actions simulation)
• Dashboard app

...sent from Jenny & Travis