herbygitea/pbs-projects

Fork 0

herbygitea 1d352fe606 Update pbsii-cicd-pipeline.md via n8n

2026-04-19 03:49:54 +00:00

9.4 KiB

Raw Blame History

project

type

status

path

PBSII CI/CD Pipeline (Revised)

Goal

Automate deployment of PBSII layers using GitHub Actions with a self-hosted runner on ustest1. Each layer deploys independently based on which files changed. Branch strategy controls target environment: main → production, staging → staging. Start with the dashboard layer, then extend to collector and parser.

Context

PBSII is a private monorepo on GitHub with subdirectories: collector/, parser/, dashboard/, n8n/
Ansible project (wp-i) handles all server config and Docker deployments, lives on ustest1
Each Docker container has its own folder and docker-compose.yml on the server (e.g., /opt/docker/dashboard/)
Ansible docker role currently deploys all containers in a loop — migrating to per-container tags
A standalone pbs-hub deploy role exists as a reference pattern
Each PBSII layer deploys differently: dashboard is Docker, collector is systemd, parser is Python/venv
n8n workflows are version-controlled JSON — no CI/CD needed

Architecture

GitHub Push (to dashboard/ on main)
  → GitHub signals self-hosted runner on ustest1
  → ustest1 already has wp-i, SSH keys, Ansible — everything it needs
  → Runs Ansible playbook with dashboard tag
  → Ansible SSHs to Linode production, deploys dashboard
  → Google Chat notification on success/failure

GitHub Push (to dashboard/ on staging)
  → Same flow, but Ansible targets staging server

Why Self-Hosted Runner

ustest1 (Proxmox homelab) already has wp-i, SSH keys to Linode, and Ansible installed
No secrets to copy into GitHub cloud runners
No cloning the wp-i repo onto temporary VMs
No rebuilding Ansible environment on every run
Solves the staging drift problem — staging deployments go through the same Ansible path as production

Phase 1: Self-Hosted Runner Setup

1a. Install GitHub Actions runner on ustest1

In the PBSII GitHub repo → Settings → Actions → Runners → New self-hosted runner:

Follow GitHub's instructions to download and configure the runner agent on ustest1
Install as a systemd service so it starts on boot
Verify runner shows as "Idle" in GitHub repo settings

1b. Runner user permissions

The runner agent runs as a user on ustest1 — that user needs SSH access to Linode (production + staging)
Ensure that user has access to wp-i on ustest1
Ensure Ansible vault password is accessible (file or environment variable)

1c. Label the runner

Add labels to the runner for targeting:

self-hosted (default)
ustest1 (custom, for clarity)

Phase 2: Ansible Dashboard Role

2a. Create the dashboard Ansible role in wp-i

Use the existing pbs-hub deploy role as a reference pattern. The role should:

Copy dashboard source files to server at /opt/docker/dashboard/
Copy docker-compose.yml and Dockerfile to the server
Run docker compose up -d --build in the dashboard folder
Healthcheck to verify dashboard is responding

2b. Tag the role

Ensure the role can be called with --tags dashboard
Verify it only touches the dashboard container, nothing else

2c. Environment targeting

The playbook needs to accept a variable (e.g., target_env) that determines whether it deploys to production or staging:

target_env: production → production Linode IP, production compose overrides
target_env: staging → staging Linode IP, staging compose overrides

This keeps the same role for both environments — only the target changes.

Phase 3: GitHub Actions Workflow (Dashboard)

3a. Production workflow

File: .github/workflows/deploy-dashboard.yml

name: Deploy Dashboard

on:
  push:
    branches: [main]
    paths:
      - 'dashboard/**'

jobs:
  deploy:
    runs-on: [self-hosted, ustest1]
    steps:
      - name: Checkout PBSII
        uses: actions/checkout@v4

      - name: Run Ansible playbook
        run: |
          cd /path/to/wp-i
          ansible-playbook playbook.yml \
            --tags dashboard \
            -e target_env=production          

      - name: Notify Google Chat (success)
        if: success()
        run: |
          curl -X POST "$GOOGLE_CHAT_WEBHOOK" \
            -H 'Content-Type: application/json' \
            -d '{"text": "✅ Dashboard deployed to production (commit: ${{          
github.sha }})" }'

      - name: Notify Google Chat (failure)
        if: failure()
        run: |
          curl -X POST "$GOOGLE_CHAT_WEBHOOK" \
            -H 'Content-Type: application/json' \
            -d '{"text": "❌ Dashboard deploy FAILED (commit: ${{ github.sha          
}})" }'
    env:
      GOOGLE_CHAT_WEBHOOK: ${{ secrets.GOOGLE_CHAT_WEBHOOK }}

3b. Staging workflow

File: .github/workflows/deploy-dashboard-staging.yml

Same as above but:

Trigger: branches: [staging]
Ansible flag: -e target_env=staging
Notification says "staging" instead of "production"

Alternatively, this could be one workflow file with a branch conditional — decide during implementation.

3c. Test the workflow

Push a small change to dashboard/ on staging branch first
Watch the Actions tab in GitHub
Verify dashboard redeploys on staging server
Verify Google Chat notification arrives
Verify pushing to other directories does NOT trigger this workflow
Once staging is solid, test the main branch → production flow

Phase 4: Extend to Other Layers

Once the dashboard pipeline is proven, repeat the pattern:

4a. Collector pipeline

File: .github/workflows/deploy-collector.yml

Trigger: paths: ['collector/**']

Ansible role needs to:

Cross-compile Go binary in CI step (or build on server)
Copy binary to server
Restart the systemd service (systemctl restart pbs-collector)
Verify collector responds on its HTTP API

Note: Go cross-compiles easily — building on ustest1 for Linux/amd64 target and shipping the binary to Linode avoids needing Go installed on production.

4b. Parser pipeline

File: .github/workflows/deploy-parser.yml

Trigger: paths: ['parser/**']