wiki-vault/Sources/Homelab/ssh-ca-with-step-ca-authentik.md

created	path	project	status	tags	type	updated
2026-05-08	Sources/Homelab	ssh-ca-step	active	authentik ansible proxmox automation	project-plan	2026-05-08

SSH CA with step-ca + Authentik

Goal

Replace long-lived SSH keypairs with short-lived certificates issued by a self-hosted CA, authenticated via Authentik OIDC. Cover human users, host identities (kill TOFU prompts), and service identities.

Architecture

• step-ca in a Debian LXC on PVE (matches Knot pattern), VLAN 11 (Lab)
• OIDC provisioner pointed at Authentik for human cert requests
• JWK provisioner for service/automation cert requests (n8n, Ansible runners, etc.)
• Host CA signs server SSH host keys → clients trust the CA, no more yes/no/fingerprint prompts
• User CA signs client certs with TTL (8h human, shorter for service)
• Ansible role rolls out TrustedUserCAKeys and HostCertificate config to fleet
• Naming: step-ca.herbylab.dev (capability-based, fits the Traefik convention)

Why this shape

• Authentik is already the SSO surface — same auth path as Traefik/MCP
• LXC keeps it lightweight and reuses Knot's deployment muscle memory
• One canary first → safer, lets us validate the Ansible role before fleet-wide
• Two CAs (user + host) is the recommended step-ca pattern; a single CA for both works but mixing principals gets messy

Phases

Phase 1 — CA foundation

• 1.1 Create LXC on PVE: Debian 13, static IP on VLAN 11, hostname step-ca
• 1.2 Install step-ca + step CLI from Smallstep apt repo
• 1.3 Initialize CA: step ca init with two intermediate CAs (user, host) under one root, RSA or Ed25519
• 1.4 Persist CA secrets: password file in /etc/step-ca/, root key offline-backup to KeePassXC
• 1.5 Systemd unit, enable + start, confirm https://step-ca.herbylab.dev:9000/health responds
• 1.6 DNS record in Knot once Knot is live; interim: /etc/hosts on canary
• 1.7 PVE snapshot: phase1-ca-base

Phase 2 — OIDC provisioner (Authentik)

• 2.1 Create OAuth2/OIDC provider in Authentik for step-ca, scopes openid email profile, redirect http://127.0.0.1/sso/oauth/callback
• 2.2 Add provisioner to step-ca: step ca provisioner add authentik --type=OIDC --client-id=... --client-secret=... --configuration-endpoint=https://auth.herbylab.dev/application/o/step-ca/.well-known/openid-configuration
• 2.3 Configure principal mapping: Authentik email → Unix username (e.g., tjcherb@plantbasedsoutherner.com → travadmin); use --admin for admin emails
• 2.4 Test from tower: step ssh login → browser SSO → cert in agent
• 2.5 Verify cert contents: step ssh inspect, confirm TTL, principals, key-id

Phase 3 — Canary server (user certs only)

• 3.1 Pick canary: us-test-authy (low blast radius, already main dev surface)
• 3.2 Fetch user CA pubkey, install to /etc/ssh/ca-user.pub
• 3.3 Add TrustedUserCAKeys /etc/ssh/ca-user.pub to sshd_config, reload
• 3.4 Test: SSH from tower with cert (key-based access still works as fallback)
• 3.5 Verify auth log shows cert-based login with correct principal
• 3.6 Run for 1 week, confirm no breakage

Phase 4 — Host certs on canary

• 4.1 Issue host cert: step ssh certificate --host us-test-authy us-test-authy.herbylab.dev /etc/ssh/ssh_host_ed25519_key.pub
• 4.2 Configure sshd_config: HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub
• 4.3 On clients: add host CA to ~/.ssh/known_hosts as @cert-authority *.herbylab.dev
• 4.4 Confirm fingerprint prompt is gone on fresh client connection
• 4.5 Set up host cert renewal (systemd timer, weekly)

Phase 5 — Ansible role + fleet rollout

• 5.1 Build step-ca-client Ansible role: installs step CLI, distributes user CA pubkey, configures sshd_config, manages host cert renewal timer
• 5.2 Inventory tagging: step_ca_enrolled group
• 5.3 Roll out to PVE host, NAS, Traefik VM, Knot LXC one-by-one
• 5.4 Update ~/.ssh/config on tower + ThinkPad to use cert-based auth by default
• 5.5 Document the "new server" runbook: 1 Ansible play vs old key-shuffle dance

Phase 6 — Service identities

• 6.1 Add JWK provisioner to step-ca for automation
• 6.2 Issue service certs to Ansible runner identity (replaces deploy keys for orchestration tasks where TTL works)
• 6.3 Decide what stays as static keys (true unattended workloads where cert renewal would be fragile) vs. what moves to certs
• 6.4 Document the tier model: human → OIDC certs, semi-attended → JWK certs, fully unattended service → static scoped keys

Phase 7 — Cleanup

• 7.1 Audit ~/.ssh/ on tower + ThinkPad, archive obsolete keys to KeePassXC, delete from disk
• 7.2 Audit authorized_keys across fleet, remove keys made redundant by CA
• 7.3 Document key rotation + CA disaster recovery (root key restore from KeePassXC backup)
• 7.4 Add fish abbreviation sshlogin → step ssh login

Open questions

• Cert TTL policy: 8h for human (re-auth daily) vs 24h (less friction)? Start at 8h, relax if painful.
• Do we want session recording? (Out of scope for step-ca; would need Teleport. Probably not.)
• Host cert renewal: systemd timer per host, or centralized via Ansible cron? Per-host is more resilient.

Dependencies / blockers

• Authentik must be reachable at auth.herbylab.dev with TLS — already true
• Knot LXC blocker (separate project) doesn't block this; we can use /etc/hosts for step-ca.herbylab.dev until DNS is sorted
• Traefik project doesn't block this — step-ca exposes its own HTTPS on :9000, no ingress needed for the CA itself

Success criteria

• Fresh laptop: install step, step ssh login, immediately SSH to any fleet server with no fingerprint prompts and no copied keys
• New server provisioning: 1 Ansible play installs CA trust + host cert, server is reachable
• Old laptop revoked: step ca revoke + 8h max blast radius
• ~/.ssh/ on personal machines contains only config and known_hosts