diff --git a/Tech/Projects/pbsii-cicd-pipeline.md b/Tech/Projects/pbsii-cicd-pipeline.md new file mode 100644 index 0000000..12cfa0c --- /dev/null +++ b/Tech/Projects/pbsii-cicd-pipeline.md @@ -0,0 +1,256 @@ +--- +project: pbsii-cicd-pipeline +type: project-plan +status: active +path: Tech/Projects +tags: + - pbs + - pbsii + - github-actions + - ansible + - cicd + - docker + - deployment +created: 2026-04-18 +updated: 2026-04-18 +--- + +# PBSII CI/CD Pipeline + +## Goal + +Automate deployment of PBSII layers using GitHub Actions to trigger Ansible +playbooks. Each layer deploys independently based on which files changed. +Start with the dashboard layer as the first pipeline, then extend to +collector and parser. + +## Context + +- PBSII is a monorepo on GitHub (private) with subdirectories: +`collector/`, `parser/`, `dashboard/`, `n8n/` +- Ansible already handles all server config (tagged) and Docker deployments +(migrating to per-container tags) +- A standalone pbs-hub deploy role exists as a reference pattern +- Each layer deploys differently: dashboard is Docker, collector is +systemd, parser is Python/venv +- n8n workflows are just version-controlled JSON — no CI/CD needed + +## Architecture + +``` +GitHub Push (to dashboard/) + → GitHub Actions workflow triggers + → Checks out PBSII repo + → Installs Ansible on runner + → Runs Ansible playbook with dashboard tag + → Ansible SSHs to Linode, deploys dashboard container +``` + +One workflow file per layer. Explicit and simple — no magic detection. + +## Phase 1: SSH Deploy Key Setup + +### 1a. Generate deploy key (local machine) + +```bash +ssh-keygen -t ed25519 -C "github-actions-deploy" -f +~/.ssh/github_deploy_linode +``` + +- No passphrase (key lives encrypted in GitHub Secrets) +- This key is dedicated to GitHub Actions — not your personal key + +### 1b. Install public key on Linode + +```bash +ssh-copy-id -i ~/.ssh/github_deploy_linode.pub your_user@SERVER_IP +``` + +- [ ] Verify: `ssh -i ~/.ssh/github_deploy_linode your_user@SERVER_IP` +connects without password prompt + +### 1c. Add GitHub Secrets + +In the PBSII GitHub repo → Settings → Secrets and variables → Actions: + +- [ ] `DEPLOY_SSH_KEY` — contents of `~/.ssh/github_deploy_linode` (the +private key) +- [ ] `DEPLOY_HOST` — your Linode server IP +- [ ] `DEPLOY_USER` — your SSH username on Linode +- [ ] `ANSIBLE_VAULT_PASSWORD` — your Ansible vault password (if using +vault; skip if not) + +### 1d. Add deploy key to Ansible inventory + +Update your Ansible playbook or inventory so it can accept the key path as +a variable. The GitHub Actions workflow will pass the key location at +runtime. + +## Phase 2: Ansible Dashboard Role + +### 2a. Create the dashboard Ansible role + +This role should handle: + +- [ ] Copy dashboard source files to server (or pull from repo on server) +- [ ] Build Docker image on server (or pull pre-built) +- [ ] Restart/recreate the dashboard container via Docker Compose +- [ ] Healthcheck to verify dashboard is responding + +### 2b. Decision: Build on server vs build in CI? + +Two options: + +**Option A — Build on server (simpler, recommended for v1):** +- GitHub Actions SSHs in via Ansible +- Ansible pulls latest code on server, runs `docker compose up -d --build +dashboard` +- No Docker registry needed + +**Option B — Build in CI, push to registry:** +- GitHub Actions builds the Docker image +- Pushes to GitHub Container Registry (ghcr.io) or Docker Hub +- Ansible pulls the image on server +- More moving parts, better for multi-server setups + +Recommendation: Start with Option A. You're deploying to one server. + +### 2c. Tag the role + +Ensure the role/playbook can be called with a tag like `--tags dashboard` +so it only touches the dashboard. + +## Phase 3: GitHub Actions Workflow + +### 3a. Create workflow file + +File: `.github/workflows/deploy-dashboard.yml` + +```yaml +name: Deploy Dashboard + +on: + push: + branches: [main] + paths: + - 'dashboard/**' + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up SSH key + run: | + mkdir -p ~/.ssh + echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/deploy_key + chmod 600 ~/.ssh/deploy_key + ssh-keyscan -H ${{ secrets.DEPLOY_HOST }} >> ~/.ssh/known_hosts + + - name: Install Ansible + run: pip install ansible + + - name: Run Ansible playbook + run: | + ansible-playbook playbook.yml \ + --tags dashboard \ + --private-key ~/.ssh/deploy_key \ + -u ${{ secrets.DEPLOY_USER }} \ + -i "${{ secrets.DEPLOY_HOST }}," + env: + ANSIBLE_HOST_KEY_CHECKING: "false" +``` + +Key points: +- `paths: ['dashboard/**']` — only triggers on changes to the dashboard +directory +- The inventory is inline (`-i "HOST,"` with trailing comma) — no inventory +file needed in the PBSII repo +- Vault password step can be added if needed + +### 3b. Test the workflow + +- [ ] Push a small change to `dashboard/` on main +- [ ] Watch the Actions tab in GitHub +- [ ] Verify the dashboard redeploys on Linode +- [ ] Verify pushing to other directories does NOT trigger this workflow + +## Phase 4: Extend to Other Layers + +Once the dashboard pipeline is solid, repeat the pattern: + +### 4a. Collector pipeline + +File: `.github/workflows/deploy-collector.yml` + +Trigger: `paths: ['collector/**']` + +Ansible role needs to: +- [ ] Build the Go binary (either on server or in CI) +- [ ] Copy binary to server +- [ ] Restart the systemd service +- [ ] Verify collector is responding on its HTTP API + +Decision: Go cross-compile in CI (fast, clean) vs build on server. CI +cross-compile is easy with Go and avoids needing Go installed on Linode. + +### 4b. Parser pipeline + +File: `.github/workflows/deploy-parser.yml` + +Trigger: `paths: ['parser/**']` + +Ansible role needs to: +- [ ] Copy parser source to server +- [ ] Ensure Python venv exists with correct deps (uv sync) +- [ ] No restart needed — n8n triggers it on schedule + +## Open Questions + +- [ ] Does the PBSII repo need its own Ansible playbook, or should it hook +into your existing main Ansible project? If separate, the playbook and +roles live in the PBSII repo. If shared, GitHub Actions needs access to the +Ansible repo too. +- [ ] Where does the dashboard Docker Compose service definition live — in +the PBSII repo or in your main compose file on the server? +- [ ] Do you want deployment notifications? (e.g., GitHub Actions → Google +Chat webhook on success/failure) +- [ ] Branch strategy: deploy only from `main`, or also from a `staging` +branch to staging server? + +## Repo Structure After Implementation + +``` +pbsii/ +├── .github/ +│ └── workflows/ +│ ├── deploy-dashboard.yml +│ ├── deploy-collector.yml +│ └── deploy-parser.yml +├── ansible/ +│ ├── playbook.yml +│ └── roles/ +│ ├── dashboard/ +│ ├── collector/ +│ └── parser/ +├── collector/ +├── parser/ +├── dashboard/ +├── n8n/ +│ └── workflows/ +└── sql/ + └── schema.sql +``` + +## Key Principles + +- Each layer deploys independently — a change to the parser never touches +the dashboard +- Ansible stays the deployment executor — GitHub Actions is just the trigger +- Build on server for v1 — move to CI builds later if needed +- Dedicated deploy key — revocable, scoped, no passphrase +- Start with dashboard, prove the pattern, then extend + +...sent from Jenny & Travis \ No newline at end of file