Env File Hardening (Sidecar Pattern)

Goal

Remove the easy path to PBS secrets. Right now .env files in /opt/docker are readable by anyone with filesystem access — including pbsdeploy. If the deploy key is compromised, an attacker can cat .env and grab every API key and password instantly.

Goal: pbsdeploy can deploy containers without ever being able to read the secrets they use.

Threat Model

Primary risk: pbsdeploy SSH key leaks → attacker SSHes in → reads .env files → owns every service.
Mitigation goal: eliminate the trivial read path. Raise the bar so an attacker would need to escalate to root (no sudo on pbsdeploy), pivot accounts, or break out of a container.
Accepted residual risk: Determined attackers with deep access can still cause damage. This is about removing low-hanging fruit, not achieving zero risk.

Architecture

Sidecar injection pattern:

.env files stay owned by root:root with 600 perms.
Ansible writes them via become: yes (sudo) — pbsdeploy never touches the files directly.
A tiny init container / sidecar runs as root, reads the locked .env, and injects secrets into the main container as environment variables.
Main containers read env vars exactly as they do today — no application code changes.

Design Decisions

Sidecar over Docker Compose secrets — Compose secrets would require refactoring every app (WordPress, n8n, pbs-hub, PBSII layers) to read from /run/secrets/ files instead of env vars. Sidecar gets the security win with zero app-level changes.
Sidecar over external secret manager (Vault, etc.) — overkill for homelab/small prod scale. Revisit if we ever move to Kubernetes.
Ansible stays the deploy tool — still writes .env files, just as root with tight perms. pbsdeploy has no sudo entry, so it can't read them back.
Scope: all services on production PBS — WordPress, n8n, pbs-hub, phpMyAdmin, everything in /opt/docker. No point hardening one service if others still leak.
Staging first — test on one service on staging, then roll out across all services on staging, then move to production.

How the Permissions Work

Ansible SSHes in as pbsdeploy → uses become: yes to elevate for file writes → files created as root:root 600.
pbsdeploy can't cat them (no read perms). Can't sudo cat them either (no sudoers entry).
Docker daemon runs as root → sidecar container runs as root → can read the files and inject into main container.
Main container sees env vars, doesn't know or care where they came from.

Next Steps

Pick a non-critical service on staging to pilot the sidecar pattern
Design the sidecar container (shell script? tiny Go/Python binary? off-the-shelf image?)
Update compose file for pilot service to include sidecar
Update Ansible role to write .env as root:root 600 (not deployer group)
Test on staging — verify containers still work, verify pbsdeploy can't read files
Roll out to all staging services
Verify nothing broke in staging over a few deploys
Roll out to production
Remove deployer group's access to .env files (or the group entirely, depending on what it ends up being used for)

Open Questions

What's the right sidecar implementation? Custom shell-based init container vs. existing tool?
Which pilot service to start with?
Does anything else in /opt/docker currently rely on the deployer group having read access that we'd break by tightening perms?

SSH Login Alerting — higher priority sibling project. Even before env hardening is done, SSH alerts give us early warning if pbsdeploy is compromised.
PBSII CI/CD — same pbsdeploy / deployer setup on staging; whatever we do here applies there too.

...sent from Jenny & Travis

4.0 KiB Raw Blame History