143 lines
5.3 KiB
Markdown
143 lines
5.3 KiB
Markdown
---
|
|
project: env-file-hardening
|
|
type: project-plan
|
|
status: active
|
|
path: Tech/Projects
|
|
tags:
|
|
- pbs
|
|
- security
|
|
- docker
|
|
- ansible
|
|
- secrets
|
|
created: 2026-04-19
|
|
updated: 2026-04-19
|
|
---
|
|
|
|
# Env File Hardening
|
|
|
|
## Goal
|
|
|
|
Remove the easy path to PBS secrets. Right now `.env` files in
|
|
`/opt/docker` are readable by anyone with filesystem access — including
|
|
`pbsdeploy`. If the deploy key is compromised, an attacker can `cat .env`
|
|
and grab every API key and password instantly.
|
|
|
|
Goal: `pbsdeploy` can deploy containers without ever being able to read the
|
|
secrets they use.
|
|
|
|
## Threat Model
|
|
|
|
- **Primary risk:** `pbsdeploy` SSH key leaks → attacker SSHes in → reads
|
|
`.env` files → owns every service.
|
|
- **Mitigation goal:** eliminate the trivial read path. Raise the bar so an
|
|
attacker would need to escalate to root (no sudo on `pbsdeploy`), pivot
|
|
accounts, or break out of a container.
|
|
- **Accepted residual risk:** Determined attackers with deep access can
|
|
still cause damage. This is about removing low-hanging fruit, not achieving
|
|
zero risk.
|
|
|
|
## Two-Phase Approach
|
|
|
|
### Phase 1: Filesystem Permissions (80/20 win)
|
|
|
|
Lock `.env` files down to `root:root 0600`. The Docker daemon runs as root,
|
|
so `docker compose` still reads them fine when processing compose files.
|
|
`pbsdeploy` runs `docker compose up`, which talks to the daemon via socket
|
|
— `pbsdeploy` itself never reads `.env` directly.
|
|
|
|
**Net effect:** the trivial `cat .env` path is gone. No sidecar needed, no
|
|
app code changes, no runtime complexity.
|
|
|
|
### Phase 2: Encryption at Rest on the Server (defense in depth, deferred)
|
|
|
|
Currently secrets are encrypted in the `wp-i` repo via ansible-vault, but
|
|
land as plaintext on the server filesystem. Phase 2 keeps them encrypted on
|
|
disk and decrypts only at container start (via sidecar or similar).
|
|
|
|
**Protects against:** backup leaks, Linode snapshot exfil, accidental
|
|
exposure, offline decrypt of historical filesystem copies.
|
|
|
|
**Not urgent because:** secrets are already encrypted in the repo, Linode
|
|
account is behind 2FA, no compliance driver. This is "nice to have when
|
|
refactoring," per earlier conversation on the topic.
|
|
|
|
## Design Decisions
|
|
|
|
- **Phase 1 is just filesystem perms** — no sidecar, no Docker secrets, no
|
|
app changes. Docker daemon reads `.env` as root; `pbsdeploy` doesn't need
|
|
to.
|
|
- **Phase 2 uses sidecar pattern when we get there** — tiny init container
|
|
decrypts at runtime, injects env vars into main container, zero app code
|
|
changes. Compose secrets ruled out because it requires refactoring every
|
|
app to read from `/run/secrets/`.
|
|
- **External secret manager (Vault, etc.) ruled out** — overkill for
|
|
self-hosted prod scale. Revisit only if moving to Kubernetes.
|
|
- **Ansible stays the deploy tool** throughout — writes `.env` files as
|
|
root with tight perms. `pbsdeploy` has no sudoers entry, so it can't read
|
|
them back.
|
|
- **Staging first** for both phases.
|
|
|
|
## How the Permissions Work (Phase 1)
|
|
|
|
- Ansible SSHes in as `pbsdeploy` → uses `become: yes` to elevate for file
|
|
writes → files created as `root:root 0600`.
|
|
- `pbsdeploy` can't `cat` them (no read perms). Can't `sudo cat` them
|
|
either (no sudoers entry).
|
|
- Docker daemon runs as root → reads `.env` when processing compose files →
|
|
injects vars into containers.
|
|
- Main container sees env vars, doesn't know or care about file perms.
|
|
|
|
## Next Steps (Phase 1)
|
|
|
|
- [ ] Audit current state — which services have `.env` files in
|
|
`/opt/docker`, current ownership and perms
|
|
- [ ] Update Ansible role(s) to write `.env` as `root:root 0600` (not
|
|
`deployer` group)
|
|
- [ ] Pick a non-critical pilot service on staging
|
|
- [ ] Deploy change to pilot, verify container still works, verify
|
|
`pbsdeploy` can't read the file
|
|
- [ ] Roll out to all staging services
|
|
- [ ] Verify nothing breaks over a few deploys
|
|
- [ ] Roll out to production
|
|
- [ ] Remove `deployer` group read access to `.env` files (group can still
|
|
exist for other things if needed)
|
|
|
|
## Next Steps (Phase 2 — later)
|
|
|
|
- [ ] Decide on encryption approach: custom sidecar vs. SOPS + age vs.
|
|
something else
|
|
- [ ] Design sidecar container (Dockerfile, entrypoint, decryption strategy)
|
|
- [ ] Decide where the decryption key lives on the server and how it's
|
|
protected
|
|
- [ ] Pilot on one staging service
|
|
- [ ] Roll out across stack
|
|
|
|
## Open Questions
|
|
|
|
### Phase 1
|
|
|
|
- Does anything in `/opt/docker` currently rely on `deployer` group read
|
|
access to `.env` that we'd break by tightening perms?
|
|
- Are all services using `env_file:` in compose, or do some hardcode
|
|
secrets in the compose file itself (which would need separate handling)?
|
|
|
|
### Phase 2 (deferred)
|
|
|
|
- Sidecar implementation: custom shell-based init container vs.
|
|
off-the-shelf tool like SOPS?
|
|
- Where does the decryption key live on the server? (If it's in another
|
|
root-owned file, we're just moving the problem — though one layer deeper is
|
|
still an improvement.)
|
|
|
|
## Depends On / Related
|
|
|
|
- **SSH Login Alerting** — higher priority sibling project. Early warning
|
|
system for the threat this project mitigates.
|
|
- **PBSII CI/CD** — same `pbsdeploy` / `deployer` setup on staging; phase 1
|
|
changes apply there too.
|
|
- **Previous conversation on secrets managers** — concluded that for
|
|
single-server self-hosted prod, encryption-at-rest on the server is "nice
|
|
to have," not urgent. Phase 2 is the realization of that future work.
|
|
|
|
|
|
...sent from Jenny & Travis |