137 lines
4.7 KiB
Markdown
137 lines
4.7 KiB
Markdown
---
|
|
project: zero-check-pipeline
|
|
type: project-plan
|
|
status: active
|
|
path: Tech/Projects
|
|
tags:
|
|
- homelab
|
|
- devops
|
|
- automation
|
|
- claude-code
|
|
created: 2026-04-18
|
|
updated: 2026-04-18
|
|
---
|
|
|
|
## Zero-Check Validation Pipeline
|
|
|
|
A reusable, language-aware validation pipeline for post-LLM code
|
|
generation. Treats all generated code as untrusted and requires it to prove
|
|
safety before promotion.
|
|
|
|
### Core Principles
|
|
|
|
- LLM-agnostic — works on code from Claude, Gemini, or any generator
|
|
- Language-aware — detects Python and Go, runs appropriate tool chain
|
|
- Read-only checks — the gauntlet scans and reports, never modifies
|
|
- Scripts are locked — the LLM can fix code but cannot weaken the checks
|
|
- Phased commits — commit at phase boundaries for rollback granularity
|
|
- Git is the notebook, the gauntlet is the quality gate
|
|
|
|
### Pipeline Flow
|
|
|
|
**Phase 1 — Code Generation**
|
|
- LLM generates code on Ubuntu server (us-test-authy)
|
|
- Commit at phase boundaries as work progresses
|
|
|
|
**Phase 2 — Gauntlet Loop (automated, max 3 iterations)**
|
|
- Secret detection (gitleaks)
|
|
- Linting (ruff for Python, golangci-lint for Go)
|
|
- SAST (semgrep with OWASP + language rulesets)
|
|
- Dependency audit (pip-audit for Python, govulncheck for Go)
|
|
- Unit tests (pytest for Python, go test for Go)
|
|
- Output is structured JSON so the LLM can parse failures and fix
|
|
- Each iteration gets a commit for traceability
|
|
- If not green after 3 attempts, escalate to human
|
|
|
|
**Phase 3 — Container Verification (one-shot, not in the loop)**
|
|
- Build from Dockerfile in clean environment
|
|
- Run full test suite inside container
|
|
- Docker Compose for Tier 2 integration tests (databases, Redis, etc.)
|
|
- Failure here gets kicked back to LLM as a specific task
|
|
|
|
**Phase 4 — Architect Sweep (human, ~5 minutes)**
|
|
- Review git diff from last clean baseline
|
|
- Check for outbound calls (requests, httpx, urllib)
|
|
- Check for auth bypasses (try/except: pass, commented-out decorators)
|
|
- Check for obfuscation (unexplained Base64/Hex strings)
|
|
- Approve or reject
|
|
|
|
### Testing Tiers
|
|
|
|
**Tier 1 — Self-contained (automated loop handles this)**
|
|
- Linting, SAST, secrets, dependency audit, unit tests with mocks
|
|
- No network, no services, no auth required
|
|
|
|
**Tier 2 — Local services (container verification handles this)**
|
|
- App + database/Redis/queue via Docker Compose
|
|
- Isolated network, disposable, no real credentials
|
|
|
|
**Tier 3 — External services (manual, deferred)**
|
|
- Real OAuth, external APIs, Cloudflare integration
|
|
- Mock boundaries in automated tests, manual smoke test during architect
|
|
sweep
|
|
- Dedicated test environment is a future evolution
|
|
|
|
### Validation Artifacts
|
|
|
|
**validate/results.json** — Machine-readable pipeline state. Timestamp,
|
|
pass/fail per check, iteration count, current status. Any new Claude
|
|
session reads this to resume context.
|
|
|
|
**validate/SUMMARY.md** — Human-readable validation summary. Lives in repo,
|
|
visible in Gitea. Browsable from any device.
|
|
|
|
Both committed and pushed with the project code.
|
|
|
|
### Project Structure
|
|
|
|
```
|
|
validate/
|
|
secrets.sh # runs gitleaks
|
|
lint.sh # runs ruff or golangci-lint
|
|
sast.sh # runs semgrep
|
|
deps.sh # runs pip-audit or govulncheck
|
|
tests.sh # runs pytest or go test
|
|
run-all.sh # calls each in order, stops on failure
|
|
results.json # machine-readable output (generated)
|
|
SUMMARY.md # human-readable output (generated)
|
|
```
|
|
|
|
Each script is 10-20 lines, single purpose, human-readable.
|
|
|
|
### Orchestration
|
|
|
|
- Claude Code is the meta-controller (the brain)
|
|
- Skills define the playbook (what to run, in what order, retry rules)
|
|
- Scripts are the hands (each runs one tool, returns structured output)
|
|
- No separate app needed — Claude Code + skills + scripts covers it
|
|
|
|
### Build Order
|
|
|
|
- [ ] Individual validation scripts (secrets, lint, sast, deps, tests)
|
|
- [ ] run-all.sh orchestrator with language detection
|
|
- [ ] Structured JSON output (results.json)
|
|
- [ ] Human-readable summary generation (SUMMARY.md)
|
|
- [ ] Claude Code skill for automated gauntlet loop with retry logic
|
|
- [ ] Dockerfile template for container verification
|
|
- [ ] Architect sweep checklist document
|
|
- [ ] Docker Compose template for Tier 2 integration tests
|
|
|
|
### Evolution Path
|
|
|
|
- **Current (v1):** Shell scripts + Claude Code skill, run locally
|
|
- **v2:** Taskfile (YAML-based task runner) if orchestration logic gets
|
|
complex
|
|
- **v3:** GitHub Actions as safety net on push
|
|
- **Future:** Dashboard app for cross-project validation history (dog-food
|
|
candidate for this pipeline)
|
|
|
|
### Deferred
|
|
|
|
- GitHub Actions integration
|
|
- Tier 3 external integration test environment
|
|
- Reporting/trend dashboards
|
|
- act (local GitHub Actions simulation)
|
|
- Dashboard app
|
|
|
|
...sent from Jenny & Travis |