Add repo-hygiene-audit skill (Phase 7: repeatable hygiene sweep)
Read-only scripted audit across homelab/pbs/docker/root domains: git auditability + .git ownership/dubious, .env perms (0660/devprojects), inline compose secrets, non-git dirs, uv-init stubs, compose naming, master-vs-main default, dirty trees, worktrees, herbygitea residue, doc presence. Emits text/json/md findings mapped to remediation-plan phases. SKILL.md + scripts/audit.py (stdlib-only) + references + packed .skill. First run: 39 repos, 62 findings, ruff-clean.
This commit is contained in:
commit
236616514f
28
.gitignore
vendored
Normal file
28
.gitignore
vendored
Normal file
@ -0,0 +1,28 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
.ruff_cache/
|
||||
.mypy_cache/
|
||||
.pytest_cache/
|
||||
|
||||
# Virtual environments
|
||||
.venv/
|
||||
venv/
|
||||
env/
|
||||
|
||||
# Generated audit reports (timestamped runs are not source)
|
||||
findings-*.md
|
||||
findings-*.json
|
||||
findings-*.txt
|
||||
|
||||
# Environment / secrets
|
||||
.env
|
||||
.env.local
|
||||
.env.*.local
|
||||
|
||||
# Editors / OS
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*~
|
||||
.DS_Store
|
||||
68
CLAUDE.md
Normal file
68
CLAUDE.md
Normal file
@ -0,0 +1,68 @@
|
||||
# CLAUDE.md
|
||||
|
||||
Agent operating guide for the `repo-hygiene-audit` skill repo. Human onboarding
|
||||
lives in `README.md` — this file is the concise, gotcha-focused guide.
|
||||
|
||||
## What this is
|
||||
|
||||
A Claude skill that runs a repeatable, **read-only** hygiene audit across the
|
||||
homelab / pbs / docker / root project domains. Phase 7 of the
|
||||
repo-hygiene-remediation project: the durable version of the one-time audit.
|
||||
|
||||
## Architecture
|
||||
|
||||
- `repo-hygiene-audit/SKILL.md` — the contract: trigger description + the
|
||||
five-step workflow. Source of truth for what the skill does.
|
||||
- `repo-hygiene-audit/scripts/audit.py` — the engine. Pure stdlib, single file.
|
||||
`discover_repos()` → per-repo `CHECKS` list → `Finding` records → renderers
|
||||
(text / md / json).
|
||||
- `repo-hygiene-audit/references/` — `check-catalog.md` (every check + why),
|
||||
`findings-format.md` (output schema + category→phase map).
|
||||
- `repo-hygiene-audit.skill` — zip of `SKILL.md scripts/ references/`. Repack
|
||||
after any change under `repo-hygiene-audit/`.
|
||||
|
||||
## Gotchas / non-obvious rules
|
||||
|
||||
- **Read-only is the contract.** The script must never write to the trees it
|
||||
scans — only to `--output`. All git invocations use read subcommands. Don't
|
||||
add a check that mutates a repo.
|
||||
- **Each repo = an immediate child of a domain root.** The `root` domain is the
|
||||
exception: it names explicit repo paths (wiki-vault, wiki-context), not a
|
||||
scan root.
|
||||
- **`docker` domain repos are compose stacks** — README expected, CLAUDE.md
|
||||
intentionally NOT expected (`COMPOSE_STACK_DOMAINS`). Don't "fix" the missing
|
||||
CLAUDE.md flag suppression there.
|
||||
- **safe.directory wildcard caveat** is load-bearing in the hints: always steer
|
||||
toward a *scoped* `safe.directory <path>`, never `*`.
|
||||
- **Self-reference trap:** `audit.py` literally contains the string
|
||||
`herbygitea` as a detection pattern. `check_herbygitea_residue` excludes its
|
||||
own file by resolved path — keep that exclusion if you refactor.
|
||||
- **Heuristic checks** (`inline-secret`, `uv-init-stub`, `claude-md-bloat`) can
|
||||
false-positive. They flag *candidates* for human judgement, not certainties.
|
||||
- **Findings keys are stable** (`domain/repo:check`) on purpose — week-over-week
|
||||
JSON diffing depends on it. Don't churn check ids casually.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Run the sweep (default domains)
|
||||
python repo-hygiene-audit/scripts/audit.py
|
||||
|
||||
# Lint (matches Travis's scaffolding ruleset)
|
||||
ruff check repo-hygiene-audit/scripts/audit.py --select E,F,B,I,UP,N,S,SIM,RET,PTH
|
||||
|
||||
# Repack the .skill after editing anything under repo-hygiene-audit/
|
||||
cd repo-hygiene-audit && zip -r ../repo-hygiene-audit.skill SKILL.md scripts/ references/
|
||||
```
|
||||
|
||||
## Conventions enforced (from the remediation plan)
|
||||
|
||||
`.env` target 0660 + group `devprojects`; secrets in `.env` not inline compose;
|
||||
README = human onboarding, CLAUDE.md = concise agent guide; prefer `main` over
|
||||
`master`; GitHub-destined repos pushed manually.
|
||||
|
||||
## Don't-touch
|
||||
|
||||
- Don't add network calls or pip dependencies — stdlib-only is deliberate so it
|
||||
runs anywhere `python3` exists.
|
||||
- Don't make the audit remediate. Reporting and fixing are separate steps.
|
||||
75
README.md
Normal file
75
README.md
Normal file
@ -0,0 +1,75 @@
|
||||
# repo-hygiene-audit
|
||||
|
||||
A Claude Code / Cowork skill that runs a **repeatable, read-only hygiene audit**
|
||||
across the `homelab`, `pbs`, `docker`, and `root` project domains and reports
|
||||
the findings, grouped by remediation phase.
|
||||
|
||||
It is the durable, repeatable version of the one-time 2026-05-19/20
|
||||
documentation/repo audit — Phase 7 of the *repo-hygiene-remediation* project.
|
||||
The original sweep proved that a one-time pass isn't enough; this skill makes
|
||||
the mechanical part runnable on a cadence so the project folders don't drift
|
||||
back into inconsistency.
|
||||
|
||||
## What it checks
|
||||
|
||||
- **Git auditability & ownership** — missing/empty `.git`, dubious ownership,
|
||||
`.git`-vs-directory owner drift, group drift.
|
||||
- **Secrets & permissions** — world-readable / wrong-mode `.env` files, secrets
|
||||
inlined in compose files.
|
||||
- **Cleanups** — `uv init` leftover stubs, `docker-compose.yml` → `compose.yml`
|
||||
naming, stale review docs, `herbygitea` SSH-alias residue, stray backup dirs.
|
||||
- **Git state** — `master`-vs-`main` default branch, leftover `docs-audit`
|
||||
branches, dirty trees, stray worktrees, unpushed commits.
|
||||
- **Docs** — missing README/CLAUDE.md, oversized CLAUDE.md (likely README clone).
|
||||
|
||||
Full catalogue: [`repo-hygiene-audit/references/check-catalog.md`](repo-hygiene-audit/references/check-catalog.md).
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
cd repo-hygiene-audit
|
||||
|
||||
# Terminal triage across all default domains:
|
||||
python scripts/audit.py
|
||||
|
||||
# Phase-mapped markdown (pastes into the remediation plan):
|
||||
python scripts/audit.py --format md -o findings-$(date +%Y%m%d).md
|
||||
|
||||
# Machine-readable, for week-over-week diffing:
|
||||
python scripts/audit.py --format json -o findings-$(date +%Y%m%d).json
|
||||
```
|
||||
|
||||
No third-party dependencies — `python3` (or `uv run python`) is enough. The
|
||||
script is **read-only**: it never edits, commits, or pushes the repos it scans.
|
||||
|
||||
### Useful flags
|
||||
|
||||
| Flag | Purpose |
|
||||
|------|---------|
|
||||
| `--format {text,json,md}` | Output format (default `text`) |
|
||||
| `-o, --output PATH` | Write to a file instead of stdout |
|
||||
| `--config FILE.json` | Override the domain map |
|
||||
| `--scan NAME=PATH` | Ad-hoc scan of a directory's children |
|
||||
| `--only CATEGORY` | Restrict to one or more categories |
|
||||
| `--fail-on SEVERITY` | Non-zero exit at/above a severity (automation hook) |
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
repo-hygiene-audit-skill/
|
||||
├── README.md # this file (human onboarding)
|
||||
├── CLAUDE.md # agent operating guide
|
||||
├── repo-hygiene-audit/ # the skill itself
|
||||
│ ├── SKILL.md # trigger + workflow
|
||||
│ ├── scripts/audit.py # read-only check engine
|
||||
│ └── references/
|
||||
│ ├── check-catalog.md # every check + rationale
|
||||
│ └── findings-format.md # output schema + phase mapping
|
||||
├── repo-hygiene-audit.skill # packaged bundle (zip of the above)
|
||||
└── sessions/ # session notes
|
||||
```
|
||||
|
||||
## Cadence
|
||||
|
||||
Currently **weekly, triggered manually** by Travis (no automation yet). The
|
||||
`--fail-on` flag is the hook if/when this becomes a scheduled task.
|
||||
BIN
repo-hygiene-audit.skill
Normal file
BIN
repo-hygiene-audit.skill
Normal file
Binary file not shown.
129
repo-hygiene-audit/SKILL.md
Normal file
129
repo-hygiene-audit/SKILL.md
Normal file
@ -0,0 +1,129 @@
|
||||
---
|
||||
name: repo-hygiene-audit
|
||||
description: >
|
||||
Repeatable hygiene audit across the homelab / pbs / docker / root project
|
||||
domains. Runs scripted, read-only checks for git auditability and .git
|
||||
ownership drift, .env file permissions, inline secrets in compose files,
|
||||
non-git directories, uv-init leftover stubs, docker-compose.yml -> compose.yml
|
||||
naming, branch and master-vs-main default-branch state, dirty trees, stray
|
||||
worktrees, and herbygitea SSH-alias residue — then emits a findings report
|
||||
mapped to the remediation-plan phases. Use when asked to audit repos, run a
|
||||
repo-hygiene sweep, check project folders for drift, do the weekly/recurring
|
||||
guided review, or verify repos against conventions before a commit/merge.
|
||||
---
|
||||
|
||||
# Repo Hygiene Audit
|
||||
|
||||
Makes the one-time 2026-05-19/20 documentation/repo audit **repeatable** so the
|
||||
project folders don't silently drift back into inconsistency. The mechanical
|
||||
checks are scripted (`scripts/audit.py`); the judgement calls (semantic doc
|
||||
freshness, policy decisions) stay with the human/agent driving the review.
|
||||
|
||||
This skill is **read-only against the trees it scans.** It never edits, commits,
|
||||
or pushes anything in the audited repos — it reports. Remediation is a separate,
|
||||
deliberate step the operator drives.
|
||||
|
||||
## When to run
|
||||
|
||||
- The recurring guided review (current cadence: **weekly, manually triggered**).
|
||||
- After landing a batch of new repos or scaffolds, to confirm they start clean.
|
||||
- Before a commit/merge sweep, to catch state that should be resolved first.
|
||||
- Whenever asked to "audit repos", "check for drift", "repo hygiene sweep".
|
||||
|
||||
## The method
|
||||
|
||||
The original audit scored docs on a five-dimension rubric — **accuracy
|
||||
(heaviest) → freshness → internal contradiction → completeness → role fit** —
|
||||
and verified documented commands/paths against the real tree (a confidently
|
||||
wrong doc is worse than a missing one). That semantic scoring is the human part.
|
||||
This skill automates the **mechanical enumeration** underneath it: sweep every
|
||||
repo in every domain and flag the objective signals that drift produces.
|
||||
|
||||
Domains (the four the audit covered):
|
||||
|
||||
| Domain | Roots scanned |
|
||||
|--------|---------------|
|
||||
| `homelab` | `/opt/projects/homelab/*` |
|
||||
| `pbs` | `/opt/projects/pbs/*` |
|
||||
| `docker` | `/opt/projects/docker/*`, `~/projects/docker/*` (compose stacks) |
|
||||
| `root` | `/opt/projects/wiki-vault`, `/opt/projects/wiki-context` |
|
||||
|
||||
`docker` repos are treated as compose stacks: README expected, **no per-stack
|
||||
CLAUDE.md** (per the locked doc-role convention).
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1 — Run the scripted sweep
|
||||
|
||||
```bash
|
||||
cd repo-hygiene-audit
|
||||
# Human-readable, all domains (default config):
|
||||
python scripts/audit.py
|
||||
|
||||
# Phase-mapped markdown you can paste into the remediation plan:
|
||||
python scripts/audit.py --format md -o findings-$(date +%Y%m%d).md
|
||||
|
||||
# Machine-readable for diffing week-over-week:
|
||||
python scripts/audit.py --format json -o findings-$(date +%Y%m%d).json
|
||||
```
|
||||
|
||||
No third-party dependencies — plain `python3` (or `uv run python`) is enough.
|
||||
|
||||
### Step 2 — Triage by phase
|
||||
|
||||
Findings are grouped into the categories that map onto the remediation plan's
|
||||
phases (see `references/findings-format.md`):
|
||||
|
||||
- **SECURITY** (Phase 2) — world-readable / wrong-mode `.env`, inline compose secrets.
|
||||
- **AUDITABILITY** (Phase 3) — missing `.git`, dubious ownership, `.git`/dir owner drift, non-git dirs with content.
|
||||
- **CLEANUP** (Phase 4) — uv-init stubs, compose naming, stale review docs, herbygitea residue, stray backup dirs.
|
||||
- **GIT_STATE** — master-vs-main default, dirty trees, leftover `docs-audit` branches, unpushed commits, stray worktrees.
|
||||
- **DOCS** — missing README/CLAUDE.md, oversized CLAUDE.md (possible README clone).
|
||||
|
||||
Start at `high`, work down. The `INVENTORY` block also lists every repo (even
|
||||
clean ones) so you can confirm the sweep saw what you expected.
|
||||
|
||||
### Step 3 — Semantic doc pass (human judgement)
|
||||
|
||||
For each repo with stale-looking docs, apply the five-dimension rubric by hand:
|
||||
read README + CLAUDE.md against the actual tree. The script flags *candidates*
|
||||
(missing docs, bloated CLAUDE.md, stale review artifacts); it can't judge
|
||||
whether the prose is *accurate*. That's the reviewer's call.
|
||||
|
||||
### Step 4 — Decide remediation, don't auto-fix
|
||||
|
||||
The hint on each finding is the suggested remediation, **not** an action the
|
||||
skill takes. Sequence fixes through the normal flow (and the manual-push rule
|
||||
for GitHub-destined repos). For the safe.directory caveat specifically: prefer a
|
||||
**scoped** `safe.directory <path>` entry over the `*` wildcard, which disables
|
||||
the dubious-ownership safety check everywhere.
|
||||
|
||||
### Step 5 — Persist the run
|
||||
|
||||
Save the markdown report alongside the project plan (or as a vault session note)
|
||||
so the next run can diff against it. Drift shows up as new findings; remediation
|
||||
shows up as findings that disappear.
|
||||
|
||||
## Scripted checks
|
||||
|
||||
`scripts/audit.py` is the engine. Each check is read-only and resilient (one
|
||||
repo erroring never aborts the sweep). Full catalogue with rationale in
|
||||
`references/check-catalog.md`. Selected flags:
|
||||
|
||||
| Flag | Purpose |
|
||||
|------|---------|
|
||||
| `--format {text,json,md}` | output format (default `text`) |
|
||||
| `-o, --output PATH` | write report to a file instead of stdout |
|
||||
| `--config FILE.json` | override the domain map (`{domain: {scan:[], repos:[]}}`) |
|
||||
| `--scan NAME=PATH` | ad-hoc: scan immediate children of PATH as repos |
|
||||
| `--repo NAME=PATH` | ad-hoc: treat PATH itself as one repo |
|
||||
| `--only CATEGORY` | restrict output to one or more categories |
|
||||
| `--fail-on SEVERITY` | exit non-zero if any finding at/above that severity (CI/automation hook) |
|
||||
|
||||
## Available resources
|
||||
|
||||
| Resource | When to load | Purpose |
|
||||
|----------|--------------|---------|
|
||||
| `scripts/audit.py` | Every run | The read-only check engine |
|
||||
| `references/check-catalog.md` | Interpreting a finding | What each check looks for + why it matters |
|
||||
| `references/findings-format.md` | Wiring output into the plan | Output schema + category→phase mapping |
|
||||
76
repo-hygiene-audit/references/check-catalog.md
Normal file
76
repo-hygiene-audit/references/check-catalog.md
Normal file
@ -0,0 +1,76 @@
|
||||
# Check catalogue
|
||||
|
||||
Every check `scripts/audit.py` runs, what it looks for, the finding it emits,
|
||||
and why it matters. All checks are read-only. Each finding carries a `check`
|
||||
id (stable, dedup-friendly), a `category`, a `severity`, a `detail`, and a
|
||||
remediation `hint`.
|
||||
|
||||
The conventions these checks enforce come from the repo-hygiene-remediation
|
||||
plan's *Locked Decisions*: `.env` target mode **0660** + group **devprojects**;
|
||||
secrets live in `.env`, not inline in compose; README = human onboarding,
|
||||
CLAUDE.md = concise agent guide (not a README clone); docker compose stacks get
|
||||
README only; prefer `main` over `master`; GitHub-destined repos are pushed
|
||||
manually.
|
||||
|
||||
## AUDITABILITY (maps to Phase 3 — unblock un-auditable repos)
|
||||
|
||||
| check id | Looks for | Severity | Why |
|
||||
|----------|-----------|----------|-----|
|
||||
| `no-git` | A directory with real (non-dotfile) content but no `.git` | high | Can't be diffed, reviewed, or doc-audited until it's under version control. |
|
||||
| `no-commits` | A `.git` exists but `HEAD` has zero commits | high | Everything is untracked; needs an initial commit pass. |
|
||||
| `dubious-ownership` | `git` refuses the repo with "detected dubious ownership" | high | Repo is unusable to the current user. Fix ownership, or add a **scoped** `safe.directory <path>` — never the `*` wildcard, which disables the safety check globally. |
|
||||
| `git-owner-drift` | `.git` owner differs from the directory owner (e.g. travadmin vs herbyadmin) | medium | Symptom of mixed-user operations; precedes dubious-ownership breakage. |
|
||||
| `group-drift` | Directory group is not `devprojects` | low | Breaks the shared-group access model the dev tree relies on. |
|
||||
|
||||
## SECURITY (maps to Phase 2 — security remediation)
|
||||
|
||||
| check id | Looks for | Severity | Why |
|
||||
|----------|-----------|----------|-----|
|
||||
| `env-world-readable:<file>` | `.env` (or `.env.<env>`) readable by `other` | high | Secrets exposed to any local account. |
|
||||
| `env-mode:<file>` | `.env` mode not in {0600, 0660} | medium | Doesn't meet the 0660/0600 target. |
|
||||
| `env-group:<file>` | `.env` group is not `devprojects` | low | Group-access model drift. |
|
||||
| `inline-secret:<file>` | A compose file assigns a secret-looking key (SECRET/PASSWORD/TOKEN/KEY/JWT/ENCRYPTION/SALT…) to a **literal** value (long hex/base64, not `${VAR}` and not a placeholder) | high | Secrets belong in a gitignored `.env`, not committed in compose. Heuristic — verify before acting; rotate if the repo was ever public. |
|
||||
|
||||
`.env.example` / `.env.sample` / `.env.template` / `.env.dist` are ignored
|
||||
(they're meant to be committed and world-readable).
|
||||
|
||||
## CLEANUP (maps to Phase 4 — cross-repo cleanups)
|
||||
|
||||
| check id | Looks for | Severity | Why |
|
||||
|----------|-----------|----------|-----|
|
||||
| `uv-init-stub` | `main.py`/`hello.py` + `pyproject.toml` + (`uv.lock` or `.python-version`) in a repo with no `src/` and ≤2 `.py` files | low | Classic `uv init` residue left in non-Python repos; not on any build path. |
|
||||
| `compose-legacy-name` | `docker-compose.yml` present, no `compose.yml` | low | Modern Compose prefers `compose.yml`. |
|
||||
| `compose-rename-midflight` | Both `docker-compose.yml` and `compose.yml` present | medium | A rename was started but not finished; `git rm` the legacy file. |
|
||||
| `stale-review-doc` | `GEMINI.md` or `*review*.md` at repo root | low | Stray review artifacts; move into a `reviews/` subdir or delete. |
|
||||
| `herbygitea-residue` | The string `herbygitea` in any tracked file (the removed SSH alias) | medium | Dead remote/host alias; will break on next use. |
|
||||
| `stray-dir` | Directory name ending `.broken` / `.backup-<date>` / `.bak` / `.old` | low | Scratch/backup cruft sitting in a domain folder. |
|
||||
|
||||
## GIT_STATE (review before commit/merge)
|
||||
|
||||
| check id | Looks for | Severity | Why |
|
||||
|----------|-----------|----------|-----|
|
||||
| `master-default` | `master` branch exists, no `main` | medium | House convention is `main`. |
|
||||
| `docs-audit-branch` | A leftover `docs-audit*` branch | low | Residue from the one-time sweep; merge/push then delete. |
|
||||
| `dirty-tree` | Uncommitted changes in the working tree | low | Sequence the doc/work commit deliberately; don't audit over churn. |
|
||||
| `extra-worktrees` | More than one attached worktree | info | Possible stale worktrees to prune. |
|
||||
| `no-upstream` | Current branch has no upstream | info | Expected for unpushed audit work; informational. |
|
||||
| `unpushed-commits` | Commits ahead of upstream | info | Push when ready (manual for GitHub-destined repos). |
|
||||
|
||||
## DOCS (audit follow-up)
|
||||
|
||||
| check id | Looks for | Severity | Why |
|
||||
|----------|-----------|----------|-----|
|
||||
| `missing-readme` | No `README.md` in a git repo | medium | No human onboarding doc. |
|
||||
| `missing-claude` | No `CLAUDE.md` in a git repo (skipped for the `docker` domain) | medium | No agent operating guide. Compose stacks intentionally get README only. |
|
||||
| `claude-md-bloat` | `CLAUDE.md` > 200 lines | low | Likely a README clone — violates the role split (CLAUDE.md should be a concise, gotcha-focused agent guide). |
|
||||
|
||||
`claude-md-bloat` and the missing-doc checks flag **candidates** for the
|
||||
semantic five-dimension review; they can't judge whether the prose is accurate.
|
||||
|
||||
## Extending
|
||||
|
||||
Add a `check_*(rep: RepoReport) -> None` function that appends `Finding`
|
||||
objects, then register it in the `CHECKS` list. Keep it read-only and wrap
|
||||
risky I/O — the orchestrator already catches exceptions per check so one bad
|
||||
repo can't abort the sweep, but checks should fail soft on their own where they
|
||||
can.
|
||||
909
repo-hygiene-audit/scripts/audit.py
Normal file
909
repo-hygiene-audit/scripts/audit.py
Normal file
@ -0,0 +1,909 @@
|
||||
#!/usr/bin/env python3
|
||||
"""repo-hygiene-audit — mechanical hygiene sweep across project domains.
|
||||
|
||||
Read-only. Scans a set of domain roots, treating each immediate subdirectory
|
||||
as a repo (plus any explicit repo paths), and runs the mechanical checks that
|
||||
surfaced findings in the 2026-05-19/20 documentation/repo audit:
|
||||
|
||||
* git auditability + .git ownership drift (+ dubious-ownership detection)
|
||||
* .env file permissions (target 0660, group devprojects)
|
||||
* inline secrets committed in compose files
|
||||
* missing / broken .git, non-git directories holding real content
|
||||
* uv-init leftover stubs (hello.py/main.py + pyproject/uv.lock/.python-version)
|
||||
* docker-compose.yml -> compose.yml naming drift
|
||||
* branch state + master-vs-main default-branch mismatch
|
||||
* dirty working trees, zero-commit repos, unpushed commits, stray worktrees
|
||||
* herbygitea SSH-alias residue
|
||||
* stale review docs (GEMINI.md / *-review-*.md) and doc presence/role split
|
||||
|
||||
This script NEVER writes to the trees it scans. The only filesystem write is
|
||||
the optional findings report written to --output.
|
||||
|
||||
Pure standard library (no pip deps) so it runs anywhere uv/python3 is present.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import grp
|
||||
import json
|
||||
import os
|
||||
import pwd
|
||||
import re
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Configuration
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
# A domain either scans the immediate children of one or more roots ("scan"),
|
||||
# or names explicit repo paths ("repos"), or both. Override with --config.
|
||||
DEFAULT_DOMAINS: dict[str, dict[str, list[str]]] = {
|
||||
"homelab": {"scan": ["/opt/projects/homelab"]},
|
||||
"pbs": {"scan": ["/opt/projects/pbs"]},
|
||||
"docker": {"scan": ["/opt/projects/docker", "/home/herbyadmin/projects/docker"]},
|
||||
"root": {"repos": ["/opt/projects/wiki-vault", "/opt/projects/wiki-context"]},
|
||||
}
|
||||
|
||||
# Conventions from the repo-hygiene-remediation plan (Locked Decisions).
|
||||
EXPECTED_GROUP = "devprojects"
|
||||
ENV_MODE_OK = {0o600, 0o660}
|
||||
PREFERRED_DEFAULT_BRANCH = "main"
|
||||
|
||||
# Repos in the "docker" domain are compose stacks: README only, no CLAUDE.md.
|
||||
COMPOSE_STACK_DOMAINS = {"docker"}
|
||||
|
||||
# Directory names we never descend into / never treat as a repo.
|
||||
SKIP_NAMES = {".git", "__pycache__", "node_modules", ".venv", "venv"}
|
||||
STRAY_DIR_RE = re.compile(r"\.broken$|\.backup-\d|\.bak$|\.old$")
|
||||
|
||||
# Secret-ish keys for the inline-compose-secret heuristic.
|
||||
SECRET_KEY_RE = re.compile(
|
||||
r"(SECRET|PASSWORD|PASSWD|TOKEN|API[_-]?KEY|PRIVATE[_-]?KEY|JWT|"
|
||||
r"ENCRYPTION|SALT|CREDENTIAL)",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
# A literal value that looks like a real secret (long hex / base64-ish),
|
||||
# as opposed to an env reference (${FOO}) or an obvious placeholder.
|
||||
SECRET_VAL_RE = re.compile(r"^[A-Za-z0-9+/=_-]{16,}$")
|
||||
PLACEHOLDER_RE = re.compile(
|
||||
r"^(changeme|change_me|example|placeholder|your[_-]|xxx+|todo|none|null|"
|
||||
r"true|false|secret|password)$",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
SEVERITY_ORDER = {"high": 0, "medium": 1, "low": 2, "info": 3}
|
||||
|
||||
# Category -> project phase (for the markdown / phase-mapped report).
|
||||
CATEGORY_PHASE = {
|
||||
"SECURITY": "Phase 2 — Security remediation",
|
||||
"AUDITABILITY": "Phase 3 — Unblock un-auditable repos",
|
||||
"CLEANUP": "Phase 4 — Cross-repo cleanups",
|
||||
"GIT_STATE": "Git state (review before commit/merge)",
|
||||
"DOCS": "Docs (audit follow-up)",
|
||||
}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Data model
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
@dataclass
|
||||
class Finding:
|
||||
domain: str
|
||||
repo: str
|
||||
path: str
|
||||
category: str
|
||||
severity: str
|
||||
check: str
|
||||
detail: str
|
||||
hint: str = ""
|
||||
|
||||
@property
|
||||
def key(self) -> str:
|
||||
return f"{self.domain}/{self.repo}:{self.check}"
|
||||
|
||||
|
||||
@dataclass
|
||||
class RepoReport:
|
||||
domain: str
|
||||
name: str
|
||||
path: str
|
||||
is_git: bool = False
|
||||
owner: str = ""
|
||||
git_owner: str = ""
|
||||
group: str = ""
|
||||
branch: str = ""
|
||||
findings: list[Finding] = field(default_factory=list)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Low-level helpers (all read-only)
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def run_git(repo: Path, *args: str) -> tuple[int, str, str]:
|
||||
"""Run a git command against repo, never prompting. Returns (rc, out, err)."""
|
||||
env = dict(os.environ, GIT_TERMINAL_PROMPT="0", GIT_PAGER="cat", PAGER="cat")
|
||||
try:
|
||||
# git is resolved from PATH and all args are internally constructed
|
||||
# (no user-supplied shell input) — read-only subcommands only.
|
||||
proc = subprocess.run( # noqa: S603
|
||||
["git", "-C", str(repo), *args], # noqa: S607
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=env,
|
||||
timeout=30,
|
||||
check=False,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError) as exc: # pragma: no cover
|
||||
return 1, "", str(exc)
|
||||
return proc.returncode, proc.stdout.strip(), proc.stderr.strip()
|
||||
|
||||
|
||||
def owner_of(path: Path) -> str:
|
||||
try:
|
||||
return pwd.getpwuid(path.stat().st_uid).pw_name
|
||||
except (KeyError, OSError):
|
||||
try:
|
||||
return str(path.stat().st_uid)
|
||||
except OSError:
|
||||
return "?"
|
||||
|
||||
|
||||
def group_of(path: Path) -> str:
|
||||
try:
|
||||
return grp.getgrgid(path.stat().st_gid).gr_name
|
||||
except (KeyError, OSError):
|
||||
try:
|
||||
return str(path.stat().st_gid)
|
||||
except OSError:
|
||||
return "?"
|
||||
|
||||
|
||||
def has_real_content(repo: Path) -> bool:
|
||||
"""True if the dir holds anything other than dotfiles/skip dirs."""
|
||||
try:
|
||||
for child in repo.iterdir():
|
||||
if child.name in SKIP_NAMES:
|
||||
continue
|
||||
if child.name.startswith("."):
|
||||
continue
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
return False
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Individual checks — each appends Finding objects to rep.findings
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def check_git_and_ownership(rep: RepoReport) -> None:
|
||||
repo = Path(rep.path)
|
||||
rep.owner = owner_of(repo)
|
||||
rep.group = group_of(repo)
|
||||
git_path = repo / ".git"
|
||||
rep.is_git = git_path.exists()
|
||||
|
||||
if rep.group not in (EXPECTED_GROUP, "?"):
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "AUDITABILITY", "low",
|
||||
"group-drift",
|
||||
f"directory group is '{rep.group}', expected '{EXPECTED_GROUP}'",
|
||||
f"chgrp -R {EXPECTED_GROUP} <repo> (and set the setgid bit)",
|
||||
)
|
||||
)
|
||||
|
||||
if not rep.is_git:
|
||||
if has_real_content(repo):
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "AUDITABILITY", "high",
|
||||
"no-git",
|
||||
"directory holds real content but has no .git",
|
||||
"git init + initial commit (gitignore .env from the start), "
|
||||
"then doc it",
|
||||
)
|
||||
)
|
||||
return
|
||||
|
||||
rep.git_owner = owner_of(git_path)
|
||||
if rep.git_owner != rep.owner:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "AUDITABILITY", "medium",
|
||||
"git-owner-drift",
|
||||
f".git owned by '{rep.git_owner}' but dir owned by '{rep.owner}'",
|
||||
"chown -R the repo to one user; prefer a scoped "
|
||||
"safe.directory entry over the '*' wildcard",
|
||||
)
|
||||
)
|
||||
|
||||
# Dubious-ownership shows up on any git invocation's stderr.
|
||||
rc, _out, err = run_git(repo, "rev-parse", "--is-inside-work-tree")
|
||||
if "dubious ownership" in err.lower():
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "AUDITABILITY", "high",
|
||||
"dubious-ownership",
|
||||
"git refuses the repo: detected dubious ownership",
|
||||
"fix ownership (chown) OR add a SCOPED "
|
||||
"'safe.directory <path>' — avoid the wildcard, it disables "
|
||||
"the safety check everywhere",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def check_branch_state(rep: RepoReport) -> None:
|
||||
if not rep.is_git:
|
||||
return
|
||||
repo = Path(rep.path)
|
||||
|
||||
rc, head, _ = run_git(repo, "symbolic-ref", "--short", "HEAD")
|
||||
if rc == 0:
|
||||
rep.branch = head
|
||||
else:
|
||||
rc2, desc, _ = run_git(repo, "rev-parse", "--short", "HEAD")
|
||||
rep.branch = f"(detached @ {desc})" if rc2 == 0 else "(unknown)"
|
||||
|
||||
# zero-commit repo (everything untracked)
|
||||
rc, count, _ = run_git(repo, "rev-list", "--count", "HEAD")
|
||||
if rc != 0:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "AUDITABILITY", "high",
|
||||
"no-commits",
|
||||
"git repo with no commits yet (everything untracked)",
|
||||
"make an initial commit pass, then audit docs",
|
||||
)
|
||||
)
|
||||
return # nothing more to inspect on an empty repo
|
||||
|
||||
# default-branch naming: prefer main over master
|
||||
_, branches_raw, _ = run_git(repo, "branch", "--format=%(refname:short)")
|
||||
branches = [b.strip() for b in branches_raw.splitlines() if b.strip()]
|
||||
if "master" in branches and "main" not in branches:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "GIT_STATE", "medium",
|
||||
"master-default",
|
||||
f"default branch is 'master' (preferred: "
|
||||
f"'{PREFERRED_DEFAULT_BRANCH}')",
|
||||
"git branch -m master main (+ update remote default if pushed)",
|
||||
)
|
||||
)
|
||||
|
||||
# leftover docs-audit branches from the one-time sweep
|
||||
audit_branches = [b for b in branches if b.startswith("docs-audit")]
|
||||
if audit_branches:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "GIT_STATE", "low",
|
||||
"docs-audit-branch",
|
||||
f"leftover audit branch(es): {', '.join(audit_branches)}",
|
||||
"merge/push then delete once Travis has reviewed",
|
||||
)
|
||||
)
|
||||
|
||||
# dirty working tree
|
||||
_, porcelain, _ = run_git(repo, "status", "--porcelain")
|
||||
dirty = [ln for ln in porcelain.splitlines() if ln.strip()]
|
||||
if dirty:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "GIT_STATE", "low",
|
||||
"dirty-tree",
|
||||
f"{len(dirty)} uncommitted change(s) in the working tree",
|
||||
"commit / stash before auditing or sequencing a doc commit",
|
||||
)
|
||||
)
|
||||
|
||||
# stray worktrees
|
||||
_, wt_raw, _ = run_git(repo, "worktree", "list", "--porcelain")
|
||||
worktrees = [ln for ln in wt_raw.splitlines() if ln.startswith("worktree ")]
|
||||
if len(worktrees) > 1:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "GIT_STATE", "info",
|
||||
"extra-worktrees",
|
||||
f"{len(worktrees)} worktrees attached",
|
||||
"git worktree prune / remove if any are stale",
|
||||
)
|
||||
)
|
||||
|
||||
# unpushed / no-upstream
|
||||
rc_up, _, _ = run_git(repo, "rev-parse", "--abbrev-ref", "@{u}")
|
||||
if rc_up != 0:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "GIT_STATE", "info",
|
||||
"no-upstream",
|
||||
f"branch '{rep.branch}' has no upstream configured",
|
||||
"expected for unpushed audit work; push when Travis is ready",
|
||||
)
|
||||
)
|
||||
else:
|
||||
_, ahead, _ = run_git(repo, "rev-list", "--count", "@{u}..HEAD")
|
||||
if ahead.isdigit() and int(ahead) > 0:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "GIT_STATE", "info",
|
||||
"unpushed-commits",
|
||||
f"{ahead} commit(s) ahead of upstream (unpushed)",
|
||||
"push when ready (manual for GitHub-destined repos)",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _iter_env_files(repo: Path) -> list[Path]:
|
||||
found: list[Path] = []
|
||||
if not repo.is_dir():
|
||||
return found
|
||||
bases = [repo, *[d for d in repo.iterdir() if d.is_dir()]]
|
||||
for base in bases:
|
||||
if base.name in SKIP_NAMES:
|
||||
continue
|
||||
try:
|
||||
for f in base.iterdir():
|
||||
if not f.is_file():
|
||||
continue
|
||||
n = f.name
|
||||
if n == ".env" or (n.startswith(".env.") and not n.endswith(
|
||||
(".example", ".sample", ".template", ".dist")
|
||||
)):
|
||||
found.append(f)
|
||||
except OSError:
|
||||
continue
|
||||
return found
|
||||
|
||||
|
||||
def check_env_perms(rep: RepoReport) -> None:
|
||||
for env in _iter_env_files(Path(rep.path)):
|
||||
try:
|
||||
st = env.stat()
|
||||
except OSError:
|
||||
continue
|
||||
mode = st.st_mode & 0o777
|
||||
grpname = group_of(env)
|
||||
rel = env.relative_to(rep.path)
|
||||
world_readable = bool(mode & 0o004)
|
||||
if world_readable:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "SECURITY", "high",
|
||||
f"env-world-readable:{rel}",
|
||||
f"{rel} is world-readable (mode {oct(mode)})",
|
||||
"chmod 0660 (or 0600) the .env file",
|
||||
)
|
||||
)
|
||||
elif mode not in ENV_MODE_OK:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "SECURITY", "medium",
|
||||
f"env-mode:{rel}",
|
||||
f"{rel} mode is {oct(mode)} (target 0660 or 0600)",
|
||||
"chmod 0660 the .env file",
|
||||
)
|
||||
)
|
||||
if grpname not in (EXPECTED_GROUP, "?") and not world_readable:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "SECURITY", "low",
|
||||
f"env-group:{rel}",
|
||||
f"{rel} group is '{grpname}', expected '{EXPECTED_GROUP}'",
|
||||
f"chgrp {EXPECTED_GROUP} the .env file",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def check_compose(rep: RepoReport) -> None:
|
||||
repo = Path(rep.path)
|
||||
legacy = list(repo.glob("docker-compose.y*ml"))
|
||||
modern = list(repo.glob("compose.y*ml"))
|
||||
|
||||
if legacy and modern:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "CLEANUP", "medium",
|
||||
"compose-rename-midflight",
|
||||
"both docker-compose.yml and compose.yml present (rename "
|
||||
"mid-flight)",
|
||||
"git rm the legacy docker-compose.yml, keep compose.yml",
|
||||
)
|
||||
)
|
||||
elif legacy:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "CLEANUP", "low",
|
||||
"compose-legacy-name",
|
||||
f"legacy '{legacy[0].name}' naming (prefer compose.yml)",
|
||||
"git mv docker-compose.yml compose.yml",
|
||||
)
|
||||
)
|
||||
|
||||
# inline-secret heuristic across whatever compose files exist
|
||||
for cf in legacy + modern:
|
||||
try:
|
||||
lines = cf.read_text(errors="replace").splitlines()
|
||||
except OSError:
|
||||
continue
|
||||
hits: list[str] = []
|
||||
for ln in lines:
|
||||
stripped = ln.strip()
|
||||
if not stripped or stripped.startswith("#"):
|
||||
continue
|
||||
# only key: value or key=value forms
|
||||
m = re.match(r"-?\s*([A-Za-z0-9_]+)\s*[:=]\s*(.+)$", stripped)
|
||||
if not m:
|
||||
continue
|
||||
key, val = m.group(1), m.group(2).strip().strip("\"'")
|
||||
if not SECRET_KEY_RE.search(key):
|
||||
continue
|
||||
if "${" in val or val.startswith("$"):
|
||||
continue # env reference — the right pattern
|
||||
if PLACEHOLDER_RE.match(val):
|
||||
continue
|
||||
if SECRET_VAL_RE.match(val):
|
||||
hits.append(key)
|
||||
if hits:
|
||||
uniq = sorted(set(hits))
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "SECURITY", "high",
|
||||
f"inline-secret:{cf.name}",
|
||||
f"{cf.name} appears to inline secret literal(s): "
|
||||
f"{', '.join(uniq)}",
|
||||
"extract to .env (gitignored); rotate if ever public",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def check_uv_stub(rep: RepoReport) -> None:
|
||||
repo = Path(rep.path)
|
||||
entry_stub = (repo / "main.py").is_file() or (repo / "hello.py").is_file()
|
||||
has_pyproject = (repo / "pyproject.toml").is_file()
|
||||
has_uv_residue = (repo / "uv.lock").is_file() or (
|
||||
repo / ".python-version"
|
||||
).is_file()
|
||||
if not (entry_stub and has_pyproject and has_uv_residue):
|
||||
return
|
||||
# Real Python project? src/ layout or a package dir or many .py files.
|
||||
if (repo / "src").is_dir():
|
||||
return
|
||||
py_files = [
|
||||
p
|
||||
for p in repo.rglob("*.py")
|
||||
if SKIP_NAMES.isdisjoint(set(p.parts)) and not p.name.startswith("test")
|
||||
]
|
||||
if len(py_files) > 2:
|
||||
return # looks like a genuine Python codebase
|
||||
residue = [
|
||||
n
|
||||
for n in ("main.py", "hello.py", "pyproject.toml", "uv.lock",
|
||||
".python-version")
|
||||
if (repo / n).exists()
|
||||
]
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "CLEANUP", "low",
|
||||
"uv-init-stub",
|
||||
"uv-init leftover stub in a non-Python repo: "
|
||||
+ ", ".join(residue),
|
||||
"sweep-remove the uv-init residue if it's not on the build path",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def check_stale_review_docs(rep: RepoReport) -> None:
|
||||
repo = Path(rep.path)
|
||||
stale: list[str] = []
|
||||
for f in repo.iterdir() if repo.is_dir() else []:
|
||||
if not f.is_file():
|
||||
continue
|
||||
n = f.name
|
||||
if n == "GEMINI.md" or re.search(r"review.*\.md$", n, re.IGNORECASE):
|
||||
if n.upper() in {"README.MD", "CLAUDE.MD"}:
|
||||
continue
|
||||
stale.append(n)
|
||||
if stale:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "CLEANUP", "low",
|
||||
"stale-review-doc",
|
||||
"stray review artifact(s) at repo root: " + ", ".join(
|
||||
sorted(stale)
|
||||
),
|
||||
"remove or move into a reviews/ subdir",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def check_herbygitea_residue(rep: RepoReport) -> None:
|
||||
repo = Path(rep.path)
|
||||
# This skill's own tree (script + docs) names the alias as a detection
|
||||
# string; never let the audit flag itself.
|
||||
skill_tree = Path(__file__).resolve().parents[2]
|
||||
if repo.resolve() == skill_tree:
|
||||
return
|
||||
hits: list[str] = []
|
||||
for p in repo.rglob("*"):
|
||||
if not p.is_file():
|
||||
continue
|
||||
if not SKIP_NAMES.isdisjoint(set(p.parts)):
|
||||
continue
|
||||
if p.stat().st_size > 1_000_000:
|
||||
continue
|
||||
try:
|
||||
text = p.read_text(errors="ignore")
|
||||
except OSError:
|
||||
continue
|
||||
if "herbygitea" in text:
|
||||
hits.append(str(p.relative_to(repo)))
|
||||
if len(hits) >= 10:
|
||||
break
|
||||
if hits:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "CLEANUP", "medium",
|
||||
"herbygitea-residue",
|
||||
"references the removed 'herbygitea' SSH alias in: "
|
||||
+ ", ".join(hits[:10]),
|
||||
"update remotes/config/scripts to the canonical gitea host",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def check_docs(rep: RepoReport) -> None:
|
||||
if not rep.is_git:
|
||||
return # non-git dirs are handled by the auditability check first
|
||||
repo = Path(rep.path)
|
||||
has_readme = (repo / "README.md").is_file()
|
||||
has_claude = (repo / "CLAUDE.md").is_file()
|
||||
|
||||
if not has_readme:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "DOCS", "medium",
|
||||
"missing-readme",
|
||||
"no README.md (human onboarding doc)",
|
||||
"add a README — human onboarding per the doc-role split",
|
||||
)
|
||||
)
|
||||
|
||||
# docker compose stacks intentionally get README only, no CLAUDE.md.
|
||||
if not has_claude and rep.domain not in COMPOSE_STACK_DOMAINS:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "DOCS", "medium",
|
||||
"missing-claude",
|
||||
"no CLAUDE.md (agent operating guide)",
|
||||
"add a CLAUDE.md from the canonical skeleton",
|
||||
)
|
||||
)
|
||||
|
||||
# doc-role-split heuristic: an oversized CLAUDE.md is often a README clone.
|
||||
if has_claude:
|
||||
try:
|
||||
n_lines = len(
|
||||
(repo / "CLAUDE.md").read_text(errors="replace").splitlines()
|
||||
)
|
||||
except OSError:
|
||||
n_lines = 0
|
||||
if n_lines > 200:
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "DOCS", "low",
|
||||
"claude-md-bloat",
|
||||
f"CLAUDE.md is {n_lines} lines — may be a README clone "
|
||||
"(role split: CLAUDE.md = concise agent guide)",
|
||||
"trim to a gotcha-focused agent guide; move onboarding "
|
||||
"prose to README",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def check_stray_dir(rep: RepoReport) -> None:
|
||||
if STRAY_DIR_RE.search(rep.name):
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "CLEANUP", "low",
|
||||
"stray-dir",
|
||||
"looks like a backup/broken scratch directory",
|
||||
"archive or delete once the live copy is confirmed good",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
CHECKS = [
|
||||
check_stray_dir,
|
||||
check_git_and_ownership,
|
||||
check_branch_state,
|
||||
check_env_perms,
|
||||
check_compose,
|
||||
check_uv_stub,
|
||||
check_stale_review_docs,
|
||||
check_herbygitea_residue,
|
||||
check_docs,
|
||||
]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Orchestration
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def discover_repos(domains: dict[str, dict[str, list[str]]]) -> list[RepoReport]:
|
||||
repos: list[RepoReport] = []
|
||||
for domain, spec in domains.items():
|
||||
for root in spec.get("scan", []):
|
||||
root_p = Path(root)
|
||||
if not root_p.is_dir():
|
||||
continue
|
||||
for child in sorted(root_p.iterdir()):
|
||||
if not child.is_dir():
|
||||
continue
|
||||
if child.name in SKIP_NAMES or child.name.startswith("."):
|
||||
continue
|
||||
repos.append(RepoReport(domain, child.name, str(child)))
|
||||
for repo_path in spec.get("repos", []):
|
||||
p = Path(repo_path)
|
||||
if p.is_dir():
|
||||
repos.append(RepoReport(domain, p.name, str(p)))
|
||||
return repos
|
||||
|
||||
|
||||
def audit(domains: dict[str, dict[str, list[str]]]) -> list[RepoReport]:
|
||||
reports = discover_repos(domains)
|
||||
for rep in reports:
|
||||
for check in CHECKS:
|
||||
try:
|
||||
check(rep)
|
||||
except Exception as exc: # noqa: BLE001 — never let one repo abort the sweep
|
||||
rep.findings.append(
|
||||
Finding(
|
||||
rep.domain, rep.name, rep.path, "DOCS", "info",
|
||||
f"check-error:{check.__name__}",
|
||||
f"check {check.__name__} raised: {exc}",
|
||||
"audit.py bug — report it",
|
||||
)
|
||||
)
|
||||
return reports
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Reporting
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def all_findings(reports: list[RepoReport]) -> list[Finding]:
|
||||
out: list[Finding] = []
|
||||
for r in reports:
|
||||
out.extend(r.findings)
|
||||
out.sort(
|
||||
key=lambda f: (
|
||||
SEVERITY_ORDER.get(f.severity, 9),
|
||||
f.category,
|
||||
f.domain,
|
||||
f.repo,
|
||||
)
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def summarize(findings: list[Finding]) -> dict[str, dict[str, int]]:
|
||||
by_sev: dict[str, int] = {}
|
||||
by_cat: dict[str, int] = {}
|
||||
for f in findings:
|
||||
by_sev[f.severity] = by_sev.get(f.severity, 0) + 1
|
||||
by_cat[f.category] = by_cat.get(f.category, 0) + 1
|
||||
return {"by_severity": by_sev, "by_category": by_cat}
|
||||
|
||||
|
||||
def render_text(reports: list[RepoReport], findings: list[Finding]) -> str:
|
||||
lines: list[str] = []
|
||||
lines.append("repo-hygiene-audit — findings")
|
||||
lines.append(f"host: {socket.gethostname()} scanned: {len(reports)} repos")
|
||||
summary = summarize(findings)
|
||||
sev = summary["by_severity"]
|
||||
lines.append(
|
||||
"severity: "
|
||||
+ ", ".join(f"{k}={sev.get(k, 0)}" for k in ("high", "medium", "low", "info"))
|
||||
+ f" total={len(findings)}"
|
||||
)
|
||||
lines.append("=" * 70)
|
||||
|
||||
# inventory line per repo (so clean repos are visible too)
|
||||
lines.append("\nINVENTORY")
|
||||
for r in sorted(reports, key=lambda x: (x.domain, x.name)):
|
||||
flag = "" if r.is_git else " [NON-GIT]"
|
||||
n = len(r.findings)
|
||||
own = f"{r.owner}:{r.group}"
|
||||
gown = f" git={r.git_owner}" if r.git_owner and r.git_owner != r.owner else ""
|
||||
lines.append(
|
||||
f" {r.domain:<8} {r.name:<32} {own:<22} "
|
||||
f"branch={r.branch or '-':<14}{gown} findings={n}{flag}"
|
||||
)
|
||||
|
||||
if not findings:
|
||||
lines.append("\nNo findings. Trees are clean. ")
|
||||
return "\n".join(lines)
|
||||
|
||||
for cat in ("SECURITY", "AUDITABILITY", "CLEANUP", "GIT_STATE", "DOCS"):
|
||||
cat_f = [f for f in findings if f.category == cat]
|
||||
if not cat_f:
|
||||
continue
|
||||
lines.append(f"\n{cat} ({CATEGORY_PHASE.get(cat, '')}) [{len(cat_f)}]")
|
||||
lines.append("-" * 70)
|
||||
for f in cat_f:
|
||||
lines.append(f" [{f.severity:<6}] {f.domain}/{f.repo}: {f.detail}")
|
||||
if f.hint:
|
||||
lines.append(f" -> {f.hint}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def render_markdown(reports: list[RepoReport], findings: list[Finding]) -> str:
|
||||
now = dt.datetime.now().strftime("%Y-%m-%d %H:%M")
|
||||
out: list[str] = []
|
||||
out.append("# Repo-hygiene audit findings")
|
||||
out.append("")
|
||||
out.append(f"_Generated {now} on {socket.gethostname()} — "
|
||||
f"{len(reports)} repos scanned._")
|
||||
out.append("")
|
||||
summary = summarize(findings)
|
||||
sev = summary["by_severity"]
|
||||
out.append(
|
||||
"**Severity:** "
|
||||
+ ", ".join(f"{k} {sev.get(k, 0)}" for k in ("high", "medium", "low", "info"))
|
||||
+ f" (total {len(findings)})"
|
||||
)
|
||||
out.append("")
|
||||
out.append("Findings are grouped by the remediation-plan phase they map to, "
|
||||
"as `- [ ]` items so they paste straight into the project plan.")
|
||||
out.append("")
|
||||
|
||||
for cat in ("SECURITY", "AUDITABILITY", "CLEANUP", "GIT_STATE", "DOCS"):
|
||||
cat_f = [f for f in findings if f.category == cat]
|
||||
if not cat_f:
|
||||
continue
|
||||
out.append(f"## {CATEGORY_PHASE.get(cat, cat)}")
|
||||
out.append("")
|
||||
for f in cat_f:
|
||||
hint = f" — _{f.hint}_" if f.hint else ""
|
||||
out.append(
|
||||
f"- [ ] `{f.domain}/{f.repo}` ({f.severity}): {f.detail}{hint}"
|
||||
)
|
||||
out.append("")
|
||||
|
||||
out.append("## Inventory")
|
||||
out.append("")
|
||||
out.append("| Domain | Repo | Owner:Group | Git | Branch | Findings |")
|
||||
out.append("|---|---|---|---|---|---|")
|
||||
for r in sorted(reports, key=lambda x: (x.domain, x.name)):
|
||||
out.append(
|
||||
f"| {r.domain} | {r.name} | {r.owner}:{r.group} | "
|
||||
f"{'yes' if r.is_git else 'NO'} | {r.branch or '-'} | "
|
||||
f"{len(r.findings)} |"
|
||||
)
|
||||
out.append("")
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def render_json(reports: list[RepoReport], findings: list[Finding]) -> str:
|
||||
payload = {
|
||||
"generated": dt.datetime.now().isoformat(timespec="seconds"),
|
||||
"host": socket.gethostname(),
|
||||
"repos_scanned": len(reports),
|
||||
"summary": summarize(findings),
|
||||
"inventory": [
|
||||
{
|
||||
"domain": r.domain,
|
||||
"repo": r.name,
|
||||
"path": r.path,
|
||||
"is_git": r.is_git,
|
||||
"owner": r.owner,
|
||||
"git_owner": r.git_owner,
|
||||
"group": r.group,
|
||||
"branch": r.branch,
|
||||
"finding_count": len(r.findings),
|
||||
}
|
||||
for r in reports
|
||||
],
|
||||
"findings": [asdict(f) | {"key": f.key} for f in findings],
|
||||
}
|
||||
return json.dumps(payload, indent=2)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# CLI
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def load_domains(args: argparse.Namespace) -> dict[str, dict[str, list[str]]]:
|
||||
if args.config:
|
||||
return json.loads(Path(args.config).read_text())
|
||||
if args.scan or args.repo:
|
||||
domains: dict[str, dict[str, list[str]]] = {}
|
||||
for item in args.scan or []:
|
||||
name, _, path = item.partition("=")
|
||||
domains.setdefault(name or "custom", {}).setdefault("scan", []).append(
|
||||
path or name
|
||||
)
|
||||
for item in args.repo or []:
|
||||
name, _, path = item.partition("=")
|
||||
domains.setdefault(name or "custom", {}).setdefault("repos", []).append(
|
||||
path or name
|
||||
)
|
||||
return domains
|
||||
return DEFAULT_DOMAINS
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
ap = argparse.ArgumentParser(
|
||||
description="Read-only repo hygiene audit across project domains."
|
||||
)
|
||||
ap.add_argument(
|
||||
"--format", choices=("text", "json", "md"), default="text",
|
||||
help="output format (default: text)",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--output", "-o", help="write report to this path (default: stdout)"
|
||||
)
|
||||
ap.add_argument(
|
||||
"--config",
|
||||
help="JSON file mapping domain -> {scan:[...], repos:[...]}",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--scan", action="append", metavar="NAME=PATH",
|
||||
help="scan immediate children of PATH as repos (repeatable)",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--repo", action="append", metavar="NAME=PATH",
|
||||
help="treat PATH itself as a repo (repeatable)",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--only", action="append", metavar="CATEGORY",
|
||||
help="only emit these categories (SECURITY/AUDITABILITY/CLEANUP/"
|
||||
"GIT_STATE/DOCS, repeatable)",
|
||||
)
|
||||
ap.add_argument(
|
||||
"--fail-on", choices=("high", "medium", "low", "info"), default=None,
|
||||
help="exit non-zero if any finding at/above this severity exists",
|
||||
)
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
domains = load_domains(args)
|
||||
reports = audit(domains)
|
||||
findings = all_findings(reports)
|
||||
|
||||
if args.only:
|
||||
wanted = {c.upper() for c in args.only}
|
||||
findings = [f for f in findings if f.category in wanted]
|
||||
|
||||
renderers = {"text": render_text, "json": render_json, "md": render_markdown}
|
||||
out = renderers[args.format](reports, findings)
|
||||
|
||||
if args.output:
|
||||
Path(args.output).write_text(out + "\n")
|
||||
print(f"wrote {args.format} report -> {args.output} "
|
||||
f"({len(findings)} findings)", file=sys.stderr)
|
||||
else:
|
||||
print(out)
|
||||
|
||||
if args.fail_on:
|
||||
threshold = SEVERITY_ORDER[args.fail_on]
|
||||
if any(SEVERITY_ORDER.get(f.severity, 9) <= threshold for f in findings):
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
39
sessions/2026-05-22-build.md
Normal file
39
sessions/2026-05-22-build.md
Normal file
@ -0,0 +1,39 @@
|
||||
# Session — build repo-hygiene-audit skill (2026-05-22)
|
||||
|
||||
Phase 7 of repo-hygiene-remediation (trellis thread "dev-setup-and-hygiene" #272):
|
||||
make the 2026-05-19/20 audit repeatable. Decisions going in: weekly cadence,
|
||||
manual trigger, full skill with scripted checks.
|
||||
|
||||
## What was built
|
||||
|
||||
- New repo `pbs/repo-hygiene-audit-skill` following the repo-per-skill house
|
||||
style (a-review-skill / session-notes-skill / Claude-Code-Scaffolding-Skill).
|
||||
- `repo-hygiene-audit/SKILL.md` — trigger description + 5-step workflow over the
|
||||
4 domains.
|
||||
- `repo-hygiene-audit/scripts/audit.py` — pure-stdlib, read-only check engine.
|
||||
- `references/check-catalog.md`, `references/findings-format.md`.
|
||||
|
||||
## Checks implemented
|
||||
|
||||
git auditability + .git ownership/dubious + group drift; `.env` perms (0660 /
|
||||
devprojects target); inline compose secrets; non-git dirs with content;
|
||||
uv-init stubs; docker-compose.yml→compose.yml naming; master-vs-main default;
|
||||
leftover docs-audit branches; dirty trees; stray worktrees; unpushed/no-upstream;
|
||||
herbygitea residue; stale review docs; missing/bloated docs.
|
||||
|
||||
## First-run results (39 repos, 4 domains)
|
||||
|
||||
high=11, medium=19, low=23, info=9 (total 62). Matched the known audit backlog:
|
||||
root-owned non-git docker stacks (authelia/traefik), world-readable .env
|
||||
(trellis-mcp, vault-mcp, ob1-deploy, python-uv .env.local), master defaults
|
||||
(authentik, pbsii, zero-check-pipeline, docker-container, ssh-login-alerter,
|
||||
hunyuan3d-sunnie, instamesh-docker), uv-init stubs (zero-check-pipeline,
|
||||
wiki-context), GEMINI.md in work-index-dashboard, herbygitea residue in
|
||||
workstation-ansible/CLAUDE.md + sshkm. Ruff clean.
|
||||
|
||||
## Open for Travis
|
||||
|
||||
- Remote: not chosen. Commit is local-only. Likely GitHub-destined (manual push).
|
||||
- Second deliverable: canonical CLAUDE.md skeleton folded into
|
||||
Claude-Code-Scaffolding-Skill (template only; not wired into scaffold.py).
|
||||
- Cadence automation deferred (`--fail-on` is the hook when ready).
|
||||
Loading…
Reference in New Issue
Block a user