PBSII CI/CD Pipeline — Session Notes (April 18, 2026)

What We Built

CI/CD pipeline for the PBSII monorepo using GitHub Actions with a self-hosted runner. Push to a branch triggers Ansible to deploy the correct layer to the correct environment.

Architecture Decisions

Monorepo stays: PBSII keeps all layers in one repo (collector/, parser/, dashboard/, n8n/). Each subdirectory is self-contained with its own deps, Dockerfile (if applicable), and README.
One workflow per layer: Separate .github/workflows/deploy-*.yml files, each triggered by paths: filter on its directory. Explicit over clever.
Self-hosted runner on ustest1: Runner agent installed as systemd service on the Proxmox homelab VM. ustest1 already has wp-i (Ansible project), SSH keys, and Ansible installed — no secrets in GitHub cloud, no rebuilding environments.
Branch strategy: staging branch → deploys to staging server. main branch → deploys to production. Solves the staging drift problem.
Ansible stays the executor: GitHub Actions is just the trigger. Ansible roles live in wp-i, source code lives in PBSII repo.
Build on server for v1: Ansible SSHs to target, pulls code, runs docker compose up -d --build. No Docker registry needed yet.
Google Chat notifications: Success/failure on every deploy.

What's Done

GitHub Actions self-hosted runner installed on ustest1 (systemd service, runs as herbyadmin)
Runner shows as Idle in GitHub repo settings
pbsdeploy user created on staging server
deployer group created, owns /opt/docker with setgid bit
pbsdeploy added to deployer and docker groups on staging
SSH key (linode_deploy) generated on ustest1, installed on staging for pbsdeploy
SSH alias spbs-runner configured on ustest1 for staging access
Production cleaned up — no deploy user, key removed
Workflow files created in PBSII repo (Lovebug)
Ansible dashboard role created in wp-i (Lovebug)
Google Chat notification workflow added (Lovebug)

What's Next

Test the pipeline: Push a small change to dashboard/ on the staging branch, watch the Actions tab — this is the moment of truth
Fix vault files: wp-i vault files (group/host vars) didn't get pushed before leaving home. Need to recreate or push from tower. Other Docker services won't work on staging until this is resolved — dashboard test may be blocked by this.
Extend to collector and parser: Same pattern — new workflow file per layer, new Ansible role per layer
Production deploy user: Once staging pipeline is proven, repeat pbsdeploy + deployer setup on production and create the main branch workflow
Security hardening (separate project): Env files with API keys/passwords are readable by anyone with Docker group access. Not a PBSII problem, but flagged.

Key Learnings

Self-hosted runner avoids the two-repo problem: Using GitHub's cloud runners would require cloning both PBSII and wp-i onto a temporary VM, exposing vault secrets. The self-hosted runner on ustest1 sidesteps all of that.
docker group is effectively root: Any user in the docker group can mount the host filesystem via a privileged container. Accepted risk for now, but worth knowing.
setgid bit (chmod g+s or chmod 2xxx): Ensures new files in /opt/docker inherit the deployer group. Without it, Ansible-created files would belong to pbsdeploy's primary group and break group permissions.
Staging-first discipline matters: We accidentally set up production first and had to backtrack. The branch strategy only works if you actually follow it.
pbs-hub PBSII layer is leaving: The override/control page will fold into the existing pbs-hub project, not live in PBSII.

Server Setup Reference

Staging server

User: pbsdeploy
Group: deployer (owns /opt/docker)
Docker group membership: yes
SSH key: linode_deploy (from ustest1)
No sudo access, no system config access

ustest1 (runner)

Runner service: actions.runner.trucktrav-pbsii.pbsii-install.service
Runner user: herbyadmin
SSH alias to staging: spbs-runner
Has wp-i repo locally
Labels: self-hosted, ustest1

...sent from Jenny & Travis

4.5 KiB Raw Blame History