--- project: env-file-hardening type: project-plan status: active path: Tech/Projects tags: - pbs - security - docker - ansible - secrets created: 2026-04-19 updated: 2026-04-19 --- # Env File Hardening ## Goal Remove the easy path to PBS secrets. Right now `.env` files in `/opt/docker` are readable by anyone with filesystem access — including `pbsdeploy`. If the deploy key is compromised, an attacker can `cat .env` and grab every API key and password instantly. Goal: `pbsdeploy` can deploy containers without ever being able to read the secrets they use. ## Threat Model - **Primary risk:** `pbsdeploy` SSH key leaks → attacker SSHes in → reads `.env` files → owns every service. - **Mitigation goal:** eliminate the trivial read path. Raise the bar so an attacker would need to escalate to root (no sudo on `pbsdeploy`), pivot accounts, or break out of a container. - **Accepted residual risk:** Determined attackers with deep access can still cause damage. This is about removing low-hanging fruit, not achieving zero risk. ## Two-Phase Approach ### Phase 1: Filesystem Permissions (80/20 win) Lock `.env` files down to `root:root 0600`. The Docker daemon runs as root, so `docker compose` still reads them fine when processing compose files. `pbsdeploy` runs `docker compose up`, which talks to the daemon via socket — `pbsdeploy` itself never reads `.env` directly. **Net effect:** the trivial `cat .env` path is gone. No sidecar needed, no app code changes, no runtime complexity. ### Phase 2: Encryption at Rest on the Server (defense in depth, deferred) Currently secrets are encrypted in the `wp-i` repo via ansible-vault, but land as plaintext on the server filesystem. Phase 2 keeps them encrypted on disk and decrypts only at container start (via sidecar or similar). **Protects against:** backup leaks, Linode snapshot exfil, accidental exposure, offline decrypt of historical filesystem copies. **Not urgent because:** secrets are already encrypted in the repo, Linode account is behind 2FA, no compliance driver. This is "nice to have when refactoring," per earlier conversation on the topic. ## Design Decisions - **Phase 1 is just filesystem perms** — no sidecar, no Docker secrets, no app changes. Docker daemon reads `.env` as root; `pbsdeploy` doesn't need to. - **Phase 2 uses sidecar pattern when we get there** — tiny init container decrypts at runtime, injects env vars into main container, zero app code changes. Compose secrets ruled out because it requires refactoring every app to read from `/run/secrets/`. - **External secret manager (Vault, etc.) ruled out** — overkill for self-hosted prod scale. Revisit only if moving to Kubernetes. - **Ansible stays the deploy tool** throughout — writes `.env` files as root with tight perms. `pbsdeploy` has no sudoers entry, so it can't read them back. - **Staging first** for both phases. ## How the Permissions Work (Phase 1) - Ansible SSHes in as `pbsdeploy` → uses `become: yes` to elevate for file writes → files created as `root:root 0600`. - `pbsdeploy` can't `cat` them (no read perms). Can't `sudo cat` them either (no sudoers entry). - Docker daemon runs as root → reads `.env` when processing compose files → injects vars into containers. - Main container sees env vars, doesn't know or care about file perms. ## Next Steps (Phase 1) - [ ] Audit current state — which services have `.env` files in `/opt/docker`, current ownership and perms - [ ] Update Ansible role(s) to write `.env` as `root:root 0600` (not `deployer` group) - [ ] Pick a non-critical pilot service on staging - [ ] Deploy change to pilot, verify container still works, verify `pbsdeploy` can't read the file - [ ] Roll out to all staging services - [ ] Verify nothing breaks over a few deploys - [ ] Roll out to production - [ ] Remove `deployer` group read access to `.env` files (group can still exist for other things if needed) ## Next Steps (Phase 2 — later) - [ ] Decide on encryption approach: custom sidecar vs. SOPS + age vs. something else - [ ] Design sidecar container (Dockerfile, entrypoint, decryption strategy) - [ ] Decide where the decryption key lives on the server and how it's protected - [ ] Pilot on one staging service - [ ] Roll out across stack ## Open Questions ### Phase 1 - Does anything in `/opt/docker` currently rely on `deployer` group read access to `.env` that we'd break by tightening perms? - Are all services using `env_file:` in compose, or do some hardcode secrets in the compose file itself (which would need separate handling)? ### Phase 2 (deferred) - Sidecar implementation: custom shell-based init container vs. off-the-shelf tool like SOPS? - Where does the decryption key live on the server? (If it's in another root-owned file, we're just moving the problem — though one layer deeper is still an improvement.) ## Depends On / Related - **SSH Login Alerting** — higher priority sibling project. Early warning system for the threat this project mitigates. - **PBSII CI/CD** — same `pbsdeploy` / `deployer` setup on staging; phase 1 changes apply there too. - **Previous conversation on secrets managers** — concluded that for single-server self-hosted prod, encryption-at-rest on the server is "nice to have," not urgent. Phase 2 is the realization of that future work. ...sent from Jenny & Travis