pbs-projects/Tech/Projects/pbsii-cicd-pipeline.md

project	type	status	path	tags	created	updated
pbsii-cicd-pipeline	project-plan	active	Tech/Projects	pbs pbsii github-actions ansible cicd docker deployment	2026-04-18	2026-04-18

PBSII CI/CD Pipeline

Goal

Automate deployment of PBSII layers using GitHub Actions to trigger Ansible playbooks. Each layer deploys independently based on which files changed. Start with the dashboard layer as the first pipeline, then extend to collector and parser.

Context

• PBSII is a monorepo on GitHub (private) with subdirectories: collector/, parser/, dashboard/, n8n/
• Ansible already handles all server config (tagged) and Docker deployments (migrating to per-container tags)
• A standalone pbs-hub deploy role exists as a reference pattern
• Each layer deploys differently: dashboard is Docker, collector is systemd, parser is Python/venv
• n8n workflows are just version-controlled JSON — no CI/CD needed

Architecture

GitHub Push (to dashboard/)
  → GitHub Actions workflow triggers
  → Checks out PBSII repo
  → Installs Ansible on runner
  → Runs Ansible playbook with dashboard tag
  → Ansible SSHs to Linode, deploys dashboard container

One workflow file per layer. Explicit and simple — no magic detection.

Phase 1: SSH Deploy Key Setup

1a. Generate deploy key (local machine)

ssh-keygen -t ed25519 -C "github-actions-deploy" -f
~/.ssh/github_deploy_linode

• No passphrase (key lives encrypted in GitHub Secrets)
• This key is dedicated to GitHub Actions — not your personal key

1b. Install public key on Linode

ssh-copy-id -i ~/.ssh/github_deploy_linode.pub your_user@SERVER_IP

• Verify: ssh -i ~/.ssh/github_deploy_linode your_user@SERVER_IP connects without password prompt

1c. Add GitHub Secrets

In the PBSII GitHub repo → Settings → Secrets and variables → Actions:

• DEPLOY_SSH_KEY — contents of ~/.ssh/github_deploy_linode (the private key)
• DEPLOY_HOST — your Linode server IP
• DEPLOY_USER — your SSH username on Linode
• ANSIBLE_VAULT_PASSWORD — your Ansible vault password (if using vault; skip if not)

1d. Add deploy key to Ansible inventory

Update your Ansible playbook or inventory so it can accept the key path as a variable. The GitHub Actions workflow will pass the key location at runtime.

Phase 2: Ansible Dashboard Role

2a. Create the dashboard Ansible role

This role should handle:

• Copy dashboard source files to server (or pull from repo on server)
• Build Docker image on server (or pull pre-built)
• Restart/recreate the dashboard container via Docker Compose
• Healthcheck to verify dashboard is responding

2b. Decision: Build on server vs build in CI?

Two options:

Option A — Build on server (simpler, recommended for v1):

• GitHub Actions SSHs in via Ansible
• Ansible pulls latest code on server, runs docker compose up -d --build dashboard
• No Docker registry needed

Option B — Build in CI, push to registry:

• GitHub Actions builds the Docker image
• Pushes to GitHub Container Registry (ghcr.io) or Docker Hub
• Ansible pulls the image on server
• More moving parts, better for multi-server setups

Recommendation: Start with Option A. You're deploying to one server.

2c. Tag the role

Ensure the role/playbook can be called with a tag like --tags dashboard so it only touches the dashboard.

Phase 3: GitHub Actions Workflow

3a. Create workflow file

File: .github/workflows/deploy-dashboard.yml

name: Deploy Dashboard

on:
  push:
    branches: [main]
    paths:
      - 'dashboard/**'

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up SSH key
        run: |
          mkdir -p ~/.ssh
          echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/deploy_key
          chmod 600 ~/.ssh/deploy_key
          ssh-keyscan -H ${{ secrets.DEPLOY_HOST }} >> ~/.ssh/known_hosts          

      - name: Install Ansible
        run: pip install ansible

      - name: Run Ansible playbook
        run: |
          ansible-playbook playbook.yml \
            --tags dashboard \
            --private-key ~/.ssh/deploy_key \
            -u ${{ secrets.DEPLOY_USER }} \
            -i "${{ secrets.DEPLOY_HOST }},"          
        env:
          ANSIBLE_HOST_KEY_CHECKING: "false"

Key points:

• paths: ['dashboard/**'] — only triggers on changes to the dashboard directory
• The inventory is inline (-i "HOST," with trailing comma) — no inventory file needed in the PBSII repo
• Vault password step can be added if needed

3b. Test the workflow

• Push a small change to dashboard/ on main
• Watch the Actions tab in GitHub
• Verify the dashboard redeploys on Linode
• Verify pushing to other directories does NOT trigger this workflow

Phase 4: Extend to Other Layers

Once the dashboard pipeline is solid, repeat the pattern:

4a. Collector pipeline

File: .github/workflows/deploy-collector.yml

Trigger: paths: ['collector/**']

Ansible role needs to:

• Build the Go binary (either on server or in CI)
• Copy binary to server
• Restart the systemd service
• Verify collector is responding on its HTTP API

Decision: Go cross-compile in CI (fast, clean) vs build on server. CI cross-compile is easy with Go and avoids needing Go installed on Linode.

4b. Parser pipeline

File: .github/workflows/deploy-parser.yml

Trigger: paths: ['parser/**']

Ansible role needs to:

• Copy parser source to server
• Ensure Python venv exists with correct deps (uv sync)
• No restart needed — n8n triggers it on schedule

Open Questions

• Does the PBSII repo need its own Ansible playbook, or should it hook into your existing main Ansible project? If separate, the playbook and roles live in the PBSII repo. If shared, GitHub Actions needs access to the Ansible repo too.
• Where does the dashboard Docker Compose service definition live — in the PBSII repo or in your main compose file on the server?
• Do you want deployment notifications? (e.g., GitHub Actions → Google Chat webhook on success/failure)
• Branch strategy: deploy only from main, or also from a staging branch to staging server?

Repo Structure After Implementation

pbsii/
├── .github/
│   └── workflows/
│       ├── deploy-dashboard.yml
│       ├── deploy-collector.yml
│       └── deploy-parser.yml
├── ansible/
│   ├── playbook.yml
│   └── roles/
│       ├── dashboard/
│       ├── collector/
│       └── parser/
├── collector/
├── parser/
├── dashboard/
├── n8n/
│   └── workflows/
└── sql/
    └── schema.sql

Key Principles

• Each layer deploys independently — a change to the parser never touches the dashboard
• Ansible stays the deployment executor — GitHub Actions is just the trigger
• Build on server for v1 — move to CI builds later if needed
• Dedicated deploy key — revocable, scoped, no passphrase
• Start with dashboard, prove the pattern, then extend

...sent from Jenny & Travis