herbygitea/pbs-projects
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Create pbs-security-hardening.md via n8n

Browse Source
This commit is contained in:
herbygitea 2026-04-03 01:00:08 +00:00
parent 1b31cb574e
commit c44d2428d0
1 changed files with 410 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

410
PBS/Tech/Projects/pbs-security-hardening.md Normal file
View File

@ -0,0 +1,410 @@
---
project: pbs-security-hardening
type: project-plan
status: active
tags:
- pbs
- docker
- traefik
- security
- crowdsec
- authelia
- cloudflare
- staging
- production
- automation
- n8n
created: 2026-04-02
updated: 2026-04-02
path: PBS/Tech/Projects
---
# PBS Security Hardening - Unified Project Plan
## Project Goal
Deploy a layered security stack for PBS infrastructure across staging and
production on Linode. Two components:
1. **Crowdsec** — Perimeter security with intelligent alerting, replacing
manual Wordfence email monitoring
2. **Authelia** — SSO with 2FA for all admin/infrastructure services
**Approach:** Deploy and validate on staging first, then roll to production.
---
## Current Security Baseline
### What's in place today
- Cloudflare: Edge WAF, DNS, CDN, Bot Fight Mode
- Wordfence: WordPress-level firewall, login lockout (5 attempts), malware
scanning
- 2FA: Enabled on WordPress admin accounts
- fail2ban: Server-level SSH protection
- UFW: Firewall with Docker-aware rules in after.rules
- Cloudflare proxy disabled for n8n and Gitea subdomains
### Problems being solved
- ~20 lockout notification emails per day (noise)
- No automated intelligent blocking — manual review required
- Targeted login attempts using real admin usernames (concerning)
- No SSO — separate credentials for Portainer, Uptime Kuma, n8n, pbs-hub,
phpMyAdmin
- No confidence that attacks aren't slipping through
---
## Architecture Overview
### Traffic flow after implementation
```
Internet → Cloudflare (edge WAF/CDN)
→ Traefik
→ Crowdsec Bouncer (block/allow decision)
→ Authelia (SSO check for admin services)
→ Service (Portainer, n8n, pbs-hub, etc.)
→ WordPress (own auth, Wordfence for malware scanning only)
```
### Tiered alerting model (Crowdsec)
- **Tier 1 — Silent block:** Generic bot attempts (admin, test, wp-admin).
Automatic. No notification.
- **Tier 2 — Logged, weekly review:** Coordinated bot patterns, repeated
IPs. Crowdsec handles it.
- **Tier 3 — Immediate alert:** Real admin username targeted. Requires
WordPress auth log as additional Crowdsec data source. Notification via n8n
→ Google Chat.
- **Tier 4 — Critical alert:** Successful login from unknown IP, failed 2FA
on real account, file integrity changes.
### Services protected by Authelia
- Portainer
- Uptime Kuma
- n8n
- pbs-hub
- phpMyAdmin
- Traefik dashboard
### Services NOT behind Authelia
- WordPress (uses its own auth + Wordfence + 2FA)
---
## Phase 1: Crowdsec on Staging
### Estimated time: 3-4 hours
### Step 1.1: Deploy Crowdsec container
- [ ] Add Crowdsec service to staging docker-compose.yml
- [ ] Configure to read Traefik access logs
- [ ] Install WordPress collection (crowdsecurity/wordpress)
- [ ] Install Traefik collection (crowdsecurity/traefik)
- [ ] Verify container starts and parses logs
### Step 1.2: Install Traefik Bouncer
- [ ] Add Crowdsec Traefik bouncer plugin to Traefik config
- [ ] Configure bouncer API key
- [ ] Verify bouncer communicates with Crowdsec
- [ ] Test: manually ban an IP via `cscli`, confirm it gets blocked at
Traefik
### Step 1.3: Add WordPress auth log source
- [ ] Add WPCode snippet or lightweight plugin to log failed logins with
usernames to a file
- [ ] Mount log file into Crowdsec container
- [ ] Configure Crowdsec to parse WordPress auth log as additional
datasource
- [ ] Verify Crowdsec sees username-level login data
### Step 1.4: Configure community blocklists
- [ ] Register free Crowdsec console account
- [ ] Enroll staging instance
- [ ] Enable community blocklists
- [ ] Verify blocked IPs appear from community intelligence
### Step 1.5: Validate on staging
- [ ] Simulate brute force from test IP — confirm auto-block
- [ ] Simulate real username attempt — confirm Tier 3 detection
- [ ] Verify WordPress still accessible for normal logins
- [ ] Verify admin tools still accessible
- [ ] Check `cscli metrics` for activity
- [ ] Check `cscli alerts list` for decisions
---
## Phase 2: Crowdsec Alerting via n8n
### Estimated time: 2-3 hours
### Step 2.1: Crowdsec notification setup
- [ ] Configure Crowdsec HTTP notification plugin
- [ ] Point notifications at n8n webhook endpoint
- [ ] Define alert profiles matching tiered model:
- Tier 1: No notification (silent block)
- Tier 2: Logged only (weekly digest — future enhancement)
- Tier 3+: Immediate webhook to n8n
### Step 2.2: n8n alerting workflow
- [ ] Create n8n workflow: Crowdsec webhook → parse alert → route by
severity
- [ ] Tier 3 alerts → Google Chat `webadmin` space via Notify Travis
sub-workflow
- [ ] Tier 4 alerts → Google Chat `webadmin` space with urgent flag
- [ ] Include: IP, reason, username (if available), timestamp, action taken
- [ ] Test end-to-end: trigger alert → verify Google Chat notification
### Step 2.3: Validate alerting
- [ ] Confirm Tier 1 produces no notification
- [ ] Confirm Tier 3 sends Google Chat alert with username info
- [ ] Confirm Tier 4 sends urgent alert
- [ ] Verify no false positives from normal browsing
---
## Phase 3: Authelia on Staging
### Estimated time: 3-4 hours
### Step 3.1: Directory and secrets setup
- [ ] Create directory structure on staging: `/opt/docker/authelia/config/`
- [ ] Generate secrets: JWT, session, storage encryption, OIDC HMAC
- [ ] Generate password hashes for Travis and Jenny (argon2id)
- [ ] Create users_database.yml with both accounts
### Step 3.2: Authelia configuration
- [ ] Write configuration.yml:
- Session domain: staging domain
- Default policy: two_factor
- Storage: SQLite (local file)
- Notifier: filesystem (for staging — SMTP for production later)
- TOTP settings: SHA1, 6 digits, 30 second period
- [ ] Define access control rules:
- Portainer, Uptime Kuma, n8n, pbs-hub, phpMyAdmin, Traefik dashboard →
two_factor
- WordPress and public routes → bypass
### Step 3.3: Docker and Traefik integration
- [ ] Create Authelia docker-compose.yml
- [ ] Add Authelia to Traefik network
- [ ] Configure Traefik forwardAuth middleware pointing to Authelia
- [ ] Add middleware labels to all protected services
- [ ] Leave WordPress compose labels unchanged
### Step 3.4: Deploy and test
- [ ] Start Authelia container
- [ ] Navigate to a protected service — verify redirect to Authelia login
- [ ] Login with Travis account — verify redirect back to service
- [ ] Register TOTP device
- [ ] Verify 2FA prompt works
- [ ] Test SSO: login once, access other protected services without re-auth
- [ ] Verify WordPress login is unaffected
- [ ] Verify public site is unaffected
---
## Phase 4: Integration Testing on Staging
### Estimated time: 2 hours
### Step 4.1: Crowdsec + Authelia together
- [ ] Verify Crowdsec bouncer doesn't interfere with Authelia redirects
- [ ] Verify blocked IPs get Crowdsec block page, not Authelia login
- [ ] Test: brute force Authelia login → Crowdsec detects and blocks
- [ ] Verify request flow: Cloudflare → Traefik → Crowdsec → Authelia →
Service
### Step 4.2: Full stack validation
- [ ] All protected services accessible after Authelia login + 2FA
- [ ] WordPress admin login works independently
- [ ] Public site unaffected
- [ ] Crowdsec blocking bots silently
- [ ] Tier 3/4 alerts arriving in Google Chat
- [ ] No performance degradation
---
## Phase 5: Roll to Production
### Estimated time: 2-3 hours
### Step 5.1: Pre-production prep
- [ ] Document all staging config that worked
- [ ] Generate fresh secrets for production (never reuse staging secrets)
- [ ] Update Authelia session domain to production domain
- [ ] Switch Authelia notifier from filesystem to SMTP (Google Workspace)
- [ ] Update Crowdsec webhook URL to production n8n
### Step 5.2: Deploy Crowdsec to production
- [ ] Add Crowdsec to production docker-compose.yml
- [ ] Add Traefik bouncer plugin
- [ ] Add WordPress auth log source
- [ ] Enroll in Crowdsec console
- [ ] Verify blocking and alerting work
### Step 5.3: Deploy Authelia to production
- [ ] Add Authelia to production docker-compose.yml
- [ ] Add forwardAuth middleware to all protected services
- [ ] Verify SSO and 2FA work
- [ ] Register TOTP devices for Travis and Jenny
### Step 5.4: Post-deployment
- [ ] Monitor for 48 hours — watch for false positives
- [ ] Tune Crowdsec scenarios if needed
- [ ] Verify Wordfence lockout emails have dropped significantly
- [ ] Demote Wordfence to malware-scanning-only role
- [ ] Consider disabling Wordfence firewall features (now redundant)
---
## Phase 6: Codify in Ansible
### Estimated time: 3-4 hours
### Step 6.1: Crowdsec Ansible role
- [ ] Create Crowdsec tasks with Ansible tags (`--tags crowdsec`)
- [ ] Template docker-compose.yml with variables
- [ ] Template Crowdsec acquis.yaml (log sources)
- [ ] Store bouncer API key in ansible-vault
- [ ] Store console enrollment key in ansible-vault
### Step 6.2: Authelia Ansible role
- [ ] Create Authelia tasks with Ansible tags (`--tags authelia`)
- [ ] Template configuration.yml with variables
- [ ] Template users_database.yml
- [ ] Store all secrets in ansible-vault
- [ ] Add forwardAuth middleware labels to protected service templates
### Step 6.3: Validate
- [ ] Destroy staging and rebuild from Ansible
- [ ] Verify Crowdsec + Authelia come up correctly
- [ ] Verify alerting works after rebuild
- [ ] Verify SSO works after rebuild
---
## Wordfence Transition Plan
After Crowdsec is stable on production:
- [ ] Disable Wordfence firewall features (redundant with Crowdsec)
- [ ] Keep Wordfence for: scheduled malware scans (monthly), file integrity
checks
- [ ] Disable Wordfence login protection (Crowdsec handles this now)
- [ ] Disable lockout email notifications (the whole point!)
- [ ] Document remaining Wordfence role in tech wiki
- [ ] Long-term: when WordPress is replaced, Wordfence goes away entirely
---
## Security Maintenance Cadence
After full deployment:
### Daily (glance, 1 min)
- Check Uptime Kuma — green means good
- Respond to any Tier 3/4 Google Chat alerts
### Weekly (10 min)
- Review Crowdsec console dashboard for trends
- Check `cscli metrics` for volume patterns
- Update WordPress plugins on staging, then production
### Monthly (30 min)
- Run Wordfence malware scan
- Review WordPress user accounts
- Check Docker image updates (Traefik, WordPress, Crowdsec, Authelia)
- Review Cloudflare analytics
- Spot-check Crowdsec community blocklist enrollment
---
## Troubleshooting
### Crowdsec not blocking
```bash
# Check if bouncer is connected
cscli bouncers list
# Check if decisions exist
cscli decisions list
# Check log parsing
cscli metrics
# Manually test a ban
cscli decisions add --ip 1.2.3.4 --reason "test ban" --duration 5m
```
### Authelia not redirecting
```bash
# Check Authelia logs
docker logs authelia
# Verify Traefik middleware
docker exec traefik wget -qO- http://localhost:8080/api/http/middlewares
# Check forwardAuth is reaching Authelia
docker logs authelia | grep "authorization"
```
### False positives
```bash
# Whitelist your own IP
cscli decisions delete --ip YOUR_IP
cscli parsers install crowdsecurity/whitelists
# Add your IP to /etc/crowdsec/parsers/s02-enrich/whitelists.yaml
```
### Locked out of Authelia
```bash
# Reset user password
docker exec authelia authelia hash-password -- 'newpassword'
# Update users_database.yml with new hash
docker restart authelia
```
---
## Resources
### Crowdsec
- Official docs: https://docs.crowdsec.net/
- Traefik bouncer:
https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
- WordPress collection:
https://hub.crowdsec.net/author/crowdsecurity/collections/wordpress
- Console: https://app.crowdsec.net/
### Authelia
- Official docs: https://www.authelia.com/
- Configuration reference:
https://www.authelia.com/configuration/prologue/introduction/
- Traefik integration: https://www.authelia.com/integration/proxies/traefik/
- File-based users: https://www.authelia.com/reference/guides/passwords/
### Traefik
- ForwardAuth middleware:
https://doc.traefik.io/traefik/middlewares/http/forwardauth/
- Plugin catalog: https://plugins.traefik.io/
---
## Success Criteria
- [ ] Wordfence lockout emails drop to near zero
- [ ] Crowdsec silently blocking bot traffic at Traefik layer
- [ ] Tier 3/4 alerts arriving in Google Chat when real threats detected
- [ ] All admin services behind Authelia SSO with 2FA
- [ ] WordPress auth unaffected
- [ ] Public site unaffected
- [ ] Everything codified in Ansible and reproducible
- [ ] Daily security overhead reduced from 15+ minutes to under 1 minute
---
*Last Updated: April 2, 2026*
*Maintained by: Travis*
*Project: Plant Based Southerner Infrastructure*
...sent from Jenny & Travis