diff --git a/Tech/Projects/env-file-hardening.md b/Tech/Projects/env-file-hardening.md index 179dd3c..da79e93 100644 --- a/Tech/Projects/env-file-hardening.md +++ b/Tech/Projects/env-file-hardening.md @@ -13,7 +13,7 @@ created: 2026-04-19 updated: 2026-04-19 --- -# Env File Hardening (Sidecar Pattern) +# Env File Hardening ## Goal @@ -36,76 +36,108 @@ accounts, or break out of a container. still cause damage. This is about removing low-hanging fruit, not achieving zero risk. -## Architecture +## Two-Phase Approach -**Sidecar injection pattern:** +### Phase 1: Filesystem Permissions (80/20 win) -- `.env` files stay owned by `root:root` with `600` perms. -- Ansible writes them via `become: yes` (sudo) — `pbsdeploy` never touches -the files directly. -- A tiny init container / sidecar runs as root, reads the locked `.env`, -and injects secrets into the main container as environment variables. -- Main containers read env vars exactly as they do today — **no application -code changes.** +Lock `.env` files down to `root:root 0600`. The Docker daemon runs as root, +so `docker compose` still reads them fine when processing compose files. +`pbsdeploy` runs `docker compose up`, which talks to the daemon via socket +— `pbsdeploy` itself never reads `.env` directly. + +**Net effect:** the trivial `cat .env` path is gone. No sidecar needed, no +app code changes, no runtime complexity. + +### Phase 2: Encryption at Rest on the Server (defense in depth, deferred) + +Currently secrets are encrypted in the `wp-i` repo via ansible-vault, but +land as plaintext on the server filesystem. Phase 2 keeps them encrypted on +disk and decrypts only at container start (via sidecar or similar). + +**Protects against:** backup leaks, Linode snapshot exfil, accidental +exposure, offline decrypt of historical filesystem copies. + +**Not urgent because:** secrets are already encrypted in the repo, Linode +account is behind 2FA, no compliance driver. This is "nice to have when +refactoring," per earlier conversation on the topic. ## Design Decisions -- **Sidecar over Docker Compose secrets** — Compose secrets would require -refactoring every app (WordPress, n8n, pbs-hub, PBSII layers) to read from -`/run/secrets/` files instead of env vars. Sidecar gets the security win -with zero app-level changes. -- **Sidecar over external secret manager (Vault, etc.)** — overkill for -homelab/small prod scale. Revisit if we ever move to Kubernetes. -- **Ansible stays the deploy tool** — still writes `.env` files, just as -root with tight perms. `pbsdeploy` has no sudo entry, so it can't read them -back. -- **Scope: all services on production PBS** — WordPress, n8n, pbs-hub, -phpMyAdmin, everything in `/opt/docker`. No point hardening one service if -others still leak. -- **Staging first** — test on one service on staging, then roll out across -all services on staging, then move to production. +- **Phase 1 is just filesystem perms** — no sidecar, no Docker secrets, no +app changes. Docker daemon reads `.env` as root; `pbsdeploy` doesn't need +to. +- **Phase 2 uses sidecar pattern when we get there** — tiny init container +decrypts at runtime, injects env vars into main container, zero app code +changes. Compose secrets ruled out because it requires refactoring every +app to read from `/run/secrets/`. +- **External secret manager (Vault, etc.) ruled out** — overkill for +self-hosted prod scale. Revisit only if moving to Kubernetes. +- **Ansible stays the deploy tool** throughout — writes `.env` files as +root with tight perms. `pbsdeploy` has no sudoers entry, so it can't read +them back. +- **Staging first** for both phases. -## How the Permissions Work +## How the Permissions Work (Phase 1) - Ansible SSHes in as `pbsdeploy` → uses `become: yes` to elevate for file -writes → files created as `root:root 600`. +writes → files created as `root:root 0600`. - `pbsdeploy` can't `cat` them (no read perms). Can't `sudo cat` them either (no sudoers entry). -- Docker daemon runs as root → sidecar container runs as root → can read -the files and inject into main container. -- Main container sees env vars, doesn't know or care where they came from. +- Docker daemon runs as root → reads `.env` when processing compose files → +injects vars into containers. +- Main container sees env vars, doesn't know or care about file perms. -## Next Steps +## Next Steps (Phase 1) -- [ ] Pick a non-critical service on staging to pilot the sidecar pattern -- [ ] Design the sidecar container (shell script? tiny Go/Python binary? -off-the-shelf image?) -- [ ] Update compose file for pilot service to include sidecar -- [ ] Update Ansible role to write `.env` as `root:root 600` (not +- [ ] Audit current state — which services have `.env` files in +`/opt/docker`, current ownership and perms +- [ ] Update Ansible role(s) to write `.env` as `root:root 0600` (not `deployer` group) -- [ ] Test on staging — verify containers still work, verify `pbsdeploy` -can't read files +- [ ] Pick a non-critical pilot service on staging +- [ ] Deploy change to pilot, verify container still works, verify +`pbsdeploy` can't read the file - [ ] Roll out to all staging services -- [ ] Verify nothing broke in staging over a few deploys +- [ ] Verify nothing breaks over a few deploys - [ ] Roll out to production -- [ ] Remove `deployer` group's access to `.env` files (or the group -entirely, depending on what it ends up being used for) +- [ ] Remove `deployer` group read access to `.env` files (group can still +exist for other things if needed) + +## Next Steps (Phase 2 — later) + +- [ ] Decide on encryption approach: custom sidecar vs. SOPS + age vs. +something else +- [ ] Design sidecar container (Dockerfile, entrypoint, decryption strategy) +- [ ] Decide where the decryption key lives on the server and how it's +protected +- [ ] Pilot on one staging service +- [ ] Roll out across stack ## Open Questions -- What's the right sidecar implementation? Custom shell-based init -container vs. existing tool? -- Which pilot service to start with? -- Does anything else in `/opt/docker` currently rely on the `deployer` -group having read access that we'd break by tightening perms? +### Phase 1 + +- Does anything in `/opt/docker` currently rely on `deployer` group read +access to `.env` that we'd break by tightening perms? +- Are all services using `env_file:` in compose, or do some hardcode +secrets in the compose file itself (which would need separate handling)? + +### Phase 2 (deferred) + +- Sidecar implementation: custom shell-based init container vs. +off-the-shelf tool like SOPS? +- Where does the decryption key live on the server? (If it's in another +root-owned file, we're just moving the problem — though one layer deeper is +still an improvement.) ## Depends On / Related -- **SSH Login Alerting** — higher priority sibling project. Even before env -hardening is done, SSH alerts give us early warning if `pbsdeploy` is -compromised. -- **PBSII CI/CD** — same `pbsdeploy` / `deployer` setup on staging; -whatever we do here applies there too. +- **SSH Login Alerting** — higher priority sibling project. Early warning +system for the threat this project mitigates. +- **PBSII CI/CD** — same `pbsdeploy` / `deployer` setup on staging; phase 1 +changes apply there too. +- **Previous conversation on secrets managers** — concluded that for +single-server self-hosted prod, encryption-at-rest on the server is "nice +to have," not urgent. Phase 2 is the realization of that future work. ...sent from Jenny & Travis \ No newline at end of file