Create zero-check-pipeline.md via n8n
This commit is contained in:
parent
9404dd5e95
commit
e0c83e60ee
137
Tech/Projects/zero-check-pipeline.md
Normal file
137
Tech/Projects/zero-check-pipeline.md
Normal file
@ -0,0 +1,137 @@
|
||||
---
|
||||
project: zero-check-pipeline
|
||||
type: project-plan
|
||||
status: active
|
||||
path: Tech/Projects
|
||||
tags:
|
||||
- homelab
|
||||
- devops
|
||||
- automation
|
||||
- claude-code
|
||||
created: 2026-04-18
|
||||
updated: 2026-04-18
|
||||
---
|
||||
|
||||
## Zero-Check Validation Pipeline
|
||||
|
||||
A reusable, language-aware validation pipeline for post-LLM code
|
||||
generation. Treats all generated code as untrusted and requires it to prove
|
||||
safety before promotion.
|
||||
|
||||
### Core Principles
|
||||
|
||||
- LLM-agnostic — works on code from Claude, Gemini, or any generator
|
||||
- Language-aware — detects Python and Go, runs appropriate tool chain
|
||||
- Read-only checks — the gauntlet scans and reports, never modifies
|
||||
- Scripts are locked — the LLM can fix code but cannot weaken the checks
|
||||
- Phased commits — commit at phase boundaries for rollback granularity
|
||||
- Git is the notebook, the gauntlet is the quality gate
|
||||
|
||||
### Pipeline Flow
|
||||
|
||||
**Phase 1 — Code Generation**
|
||||
- LLM generates code on Ubuntu server (us-test-authy)
|
||||
- Commit at phase boundaries as work progresses
|
||||
|
||||
**Phase 2 — Gauntlet Loop (automated, max 3 iterations)**
|
||||
- Secret detection (gitleaks)
|
||||
- Linting (ruff for Python, golangci-lint for Go)
|
||||
- SAST (semgrep with OWASP + language rulesets)
|
||||
- Dependency audit (pip-audit for Python, govulncheck for Go)
|
||||
- Unit tests (pytest for Python, go test for Go)
|
||||
- Output is structured JSON so the LLM can parse failures and fix
|
||||
- Each iteration gets a commit for traceability
|
||||
- If not green after 3 attempts, escalate to human
|
||||
|
||||
**Phase 3 — Container Verification (one-shot, not in the loop)**
|
||||
- Build from Dockerfile in clean environment
|
||||
- Run full test suite inside container
|
||||
- Docker Compose for Tier 2 integration tests (databases, Redis, etc.)
|
||||
- Failure here gets kicked back to LLM as a specific task
|
||||
|
||||
**Phase 4 — Architect Sweep (human, ~5 minutes)**
|
||||
- Review git diff from last clean baseline
|
||||
- Check for outbound calls (requests, httpx, urllib)
|
||||
- Check for auth bypasses (try/except: pass, commented-out decorators)
|
||||
- Check for obfuscation (unexplained Base64/Hex strings)
|
||||
- Approve or reject
|
||||
|
||||
### Testing Tiers
|
||||
|
||||
**Tier 1 — Self-contained (automated loop handles this)**
|
||||
- Linting, SAST, secrets, dependency audit, unit tests with mocks
|
||||
- No network, no services, no auth required
|
||||
|
||||
**Tier 2 — Local services (container verification handles this)**
|
||||
- App + database/Redis/queue via Docker Compose
|
||||
- Isolated network, disposable, no real credentials
|
||||
|
||||
**Tier 3 — External services (manual, deferred)**
|
||||
- Real OAuth, external APIs, Cloudflare integration
|
||||
- Mock boundaries in automated tests, manual smoke test during architect
|
||||
sweep
|
||||
- Dedicated test environment is a future evolution
|
||||
|
||||
### Validation Artifacts
|
||||
|
||||
**validate/results.json** — Machine-readable pipeline state. Timestamp,
|
||||
pass/fail per check, iteration count, current status. Any new Claude
|
||||
session reads this to resume context.
|
||||
|
||||
**validate/SUMMARY.md** — Human-readable validation summary. Lives in repo,
|
||||
visible in Gitea. Browsable from any device.
|
||||
|
||||
Both committed and pushed with the project code.
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
validate/
|
||||
secrets.sh # runs gitleaks
|
||||
lint.sh # runs ruff or golangci-lint
|
||||
sast.sh # runs semgrep
|
||||
deps.sh # runs pip-audit or govulncheck
|
||||
tests.sh # runs pytest or go test
|
||||
run-all.sh # calls each in order, stops on failure
|
||||
results.json # machine-readable output (generated)
|
||||
SUMMARY.md # human-readable output (generated)
|
||||
```
|
||||
|
||||
Each script is 10-20 lines, single purpose, human-readable.
|
||||
|
||||
### Orchestration
|
||||
|
||||
- Claude Code is the meta-controller (the brain)
|
||||
- Skills define the playbook (what to run, in what order, retry rules)
|
||||
- Scripts are the hands (each runs one tool, returns structured output)
|
||||
- No separate app needed — Claude Code + skills + scripts covers it
|
||||
|
||||
### Build Order
|
||||
|
||||
- [ ] Individual validation scripts (secrets, lint, sast, deps, tests)
|
||||
- [ ] run-all.sh orchestrator with language detection
|
||||
- [ ] Structured JSON output (results.json)
|
||||
- [ ] Human-readable summary generation (SUMMARY.md)
|
||||
- [ ] Claude Code skill for automated gauntlet loop with retry logic
|
||||
- [ ] Dockerfile template for container verification
|
||||
- [ ] Architect sweep checklist document
|
||||
- [ ] Docker Compose template for Tier 2 integration tests
|
||||
|
||||
### Evolution Path
|
||||
|
||||
- **Current (v1):** Shell scripts + Claude Code skill, run locally
|
||||
- **v2:** Taskfile (YAML-based task runner) if orchestration logic gets
|
||||
complex
|
||||
- **v3:** GitHub Actions as safety net on push
|
||||
- **Future:** Dashboard app for cross-project validation history (dog-food
|
||||
candidate for this pipeline)
|
||||
|
||||
### Deferred
|
||||
|
||||
- GitHub Actions integration
|
||||
- Tier 3 external integration test environment
|
||||
- Reporting/trend dashboards
|
||||
- act (local GitHub Actions simulation)
|
||||
- Dashboard app
|
||||
|
||||
...sent from Jenny & Travis
|
||||
Loading…
Reference in New Issue
Block a user