wiki-vault/Sources/Dev/2026-06-22-pbs-prod-clone-and-branch-merge-2026-06-22.md

---
created: '2026-06-22'
path: Sources/Dev
project: pbs-membership-platform-overview
tags:
- pbs
- membership
- content-hub
- mysql
- wordpress
- ansible
- docker
- garden
- wpcode
type: session-notes
---

## Outcome

Continuation of the prior two sessions ([[pbs-membership-phases-1-2-3a-deploy]] 2026-06-19, [[pbs-content-hub-sync-2026-06-20-21]] 2026-06-21). Today's chain (2026-06-22) did two big things:

1. **Cloned production data onto the herbylab sandbox** — `wordpress` DB (52 MB), `pbs_automation` DB (153 KB), `wp-content/` tarball (1.2 GB → 1.7 GB extracted). Domain rewritten `plantbasedsoutherner.com` → `pbs.herbylab.dev` via wp-cli search-replace (4751 replacements). Sandbox is now a real prod mirror: themed Astra frontend, real recipe pages with real images, real users from prod (admin = `sheridanbf` / user_id 7), real content-hub projects.
2. **Reconciled the wordpress-install branch divergence** that had been latent since the 2026-06-19 chain — `feat/save-button-phase2` (the Phase 2 + WP-side garden-auth work) merged into `feat/content-hub-schema-sync` (the schema + deploy fixes). All in-flight work now lives on one branch.

**Workers run this session (W13 → W15, plus an inline patch I ran without a worker for the post-restore schema fix):**

- **W13** — wipe + restore wordpress + pbs_automation DBs from prod dumps. wp-cli search-replace for domain. Redis flush. Smoke-tested.
- **Inline** — re-applied `01_content_hub.j2` against the restored pbs_automation DB to bring in the column ALTERs that prod's schema lacked (outline/treatment/trello_card_id/trello_last_synced_at). pbs-api `/pbscontenthub/` went from 500 → 200.
- **W14** — extracted `wp-content/` (1.2 GB tarball, scp'd in 6.8s via tailscale). Front page went 0 → 357 KB. Surfaced two branch-divergence gaps: missing REDIS_PASSWORD + GARDEN_AUTH_SHARED_SECRET in `roles/dockers/files/wordpress/.env.j2`; `garden-auth.php` not in this branch at all (lived only on save-button-phase2). Patched .env.j2 (uncommitted, handed to Travis); restored garden-auth.php via docker-cp from the host (sandbox-only fix).
- **Lovebug inline commit** `4811479` — landed W14's .env.j2 fix on `feat/content-hub-schema-sync`.
- **W15** — merged `feat/save-button-phase2` → `feat/content-hub-schema-sync` (merge commit `34c1034`), resolving 6 real conflicts per a pre-agreed rules table. Verified all 9 W14 smoke checks pass post-merge. garden-auth.php is now ansible-managed (no longer host-artifact-dependent).

Final state:
- `feat/content-hub-schema-sync` @ `34c1034` on gitea is the canonical integration branch — has everything from W7–W15.
- `feat/save-button-phase2` preserved at `4f2d105` (merged, not deleted).
- Sandbox: prod mirror, all surfaces work, member-garden Phase 3b count card renders against the (still-seeded-empty) `pbs_garden`.

Also: **drafted + committed `clone-prod-to-sandbox.md`** to main (`cdff2c2` on gitea) — Travis's step-by-step playbook for repeating this clone procedure. Complements (doesn't replace) the existing `disaster_recovery_plan.md` and `local-back-up-setup.md`.

## Topics Covered

- **mysqldump idioms for `wordpress` + `pbs_automation`** — `--single-transaction --quick --skip-lock-tables --routines --triggers` for an InnoDB consistent-snapshot. Travis dumped both from prod (GUI for the first run; switched to `MYSQL_PWD="$MYSQL_ROOT_PASSWORD"` inside the container after the suggestion came up). Files dropped at `/opt/backups/pbs-wp/` on the dev server (not `/var/backups/pbs-prod/` like my original doc said — Travis picked a different path, I followed his lead).
- **scp's two-host syntax bounces through the client** — not host-to-host. Modern OpenSSH 9+ removed the `-3` flag because that became the default behavior with the SFTP migration. Travis's 1.5 GB scp choked partway; documented rsync `--partial --inplace` as the resumable alternative. (He stuck with scp; it eventually finished. Future cloning should use rsync.)
- **wp-cli search-replace** — installed inline into the wordpress container (ephemeral; lost on rebuild). 4751 replacements across 24 columns, including serialized blobs. `--skip-columns=guid` (keep guids on prod domain per WP convention). The Astra/Freemius "uninitialized class" warnings during search-replace are benign — they're serialized plugin objects wp-cli can't rehydrate but doesn't need to.
- **Schema gap after restore: prod is behind pbs-api master** — prod's `pbs_automation` doesn't have the scene/metadata tables or the trello_*/outline/treatment columns on `projects`. pbs-api's `create_all()` added the missing *tables* on restart but can't ALTER `projects` to add columns. Re-applied `01_content_hub.j2` (the patch is information_schema-guarded + idempotent) — `/pbscontenthub/` immediately recovered to 200. This is the exact pattern the Alembic-centralization plan would systematize.
- **wp-content tar from inside the container vs host bind mount** — Travis didn't have wp-content bind-mounted on prod, used `docker exec wordpress tar -czf - -C /var/www/html wp-content` streaming pattern. Cleaner than the DR plan's two-command "tar to /tmp + docker cp" pattern.
- **Branch divergence is the bigger story.** W12 / W14 both surfaced files that "weren't in the repo" — when verified against `git log --all`, they ALWAYS lived on a sibling branch. The hidden cost is real: every downstream worker has to re-discover what's missing and either re-implement or stop. W15's merge is what should have happened weeks ago.
- **Duplicate-YAML-key after auto-merge is an invisible footgun.** Both branches added `member_garden_repo_version` to `inventory/group_vars/herbylab/main.yml` at different line positions. Git auto-merged with no conflict (the additions were textually disjoint). YAML last-wins silently picked the older pin. The first deploy after the merge ran member-garden from `feat/phase-3a-garden-skeleton` instead of `feat/phase-3b-saved-count`. W15 caught it after deploy showed the wrong card; amended the merge commit. Future merges that touch top-level YAML keys need a sanity check like `yq` or a key-count `grep -c "^key_name:"` post-merge.

## Key Learnings

- **W13's "schema lags pbs-api" was the inverse of the content-hub-sync chain.** The 2026-06-21 chain fixed pbs-video-manager → sandbox drift; today's restore re-introduced the gap (prod schema → sandbox). Same pattern, opposite direction. The fix path was clean because we already had `01_content_hub.j2` written — re-apply against the restored DB and the columns are back.
- **wp-cli is the only safe tool for WordPress URL rewriting at scale.** The DR plan's `sed -i 's/old/new/g' backup.sql` works for non-length-changing rewrites but corrupts PHP-serialized option_values when lengths differ (which `plantbasedsoutherner.com` → `pbs.herbylab.dev` does). 4751 replacements, several inside serialized Yoast settings + Wordfence logs — wp-cli handled them faithfully; sed would have produced subtle data-corruption that wouldn't surface until specific admin pages broke. Worth normalizing.
- **W14's "garden-auth.php not in the repo" was technically correct on the active branch but factually wrong overall.** The file was on `feat/save-button-phase2`. This is the SECOND time this pattern bit a downstream worker (W12 was the first, with the env-template + Traefik labels). Pre-spawn check: `git log --all --oneline -- <suspected_file>` before assuming a file is missing.
- **Sandbox-throwaway latitude shrinks the cost of mistakes considerably.** Drop pbs_automation, drop the wp-content tree, force-rebuild images — all routine when nothing on the sandbox is worth preserving. This session ran 3 destructive operations on the sandbox (DB drops + wp-content overwrite) with zero hand-wringing because of the framing.
- **`container_compose_build: always` is a missing convention.** Every container that builds from a clone needs it; without it, `docker_compose_v2 up -d` reuses stale images. Member-garden has it (W12). pbs-api doesn't. The flag was added to `tasks/common/deploy.yml` by W12 and is available to any container — would be a small cleanup to set it for pbs-api too.
- **Sheridan Fidelman = `sheridanbf` is the canonical admin user in prod.** Useful for cross-session cookie-minting smoke tests.

## Follow-ons

### From this session specifically

- [ ] **Set `container_compose_build: always` on pbs-api** to match member-garden. Currently relies on manual `compose build --no-cache` (per W13 + W14 workarounds). Small change, prevents stale-image deploys.
- [ ] **Bake wp-cli into the wordpress container image** (or use a wp-cli sidecar). W13 + W14 both installed it inline (ephemeral). For repeatable clone-to-sandbox + future migration work, persistent wp-cli would save the inline install step.
- [ ] **Update `clone-prod-to-sandbox.md`** with the lessons learned today: (a) actual file path used was `/opt/backups/pbs-wp/`, not `/var/backups/pbs-prod/`; (b) the post-restore schema patch (re-apply 01_content_hub.j2) is required when prod schema lags pbs-api; (c) wp-cli install command for the WP container; (d) rsync vs scp guidance for the wp-content tarball. Travis is reviewing his cloned copy and will likely fold these in.
- [ ] **YAML duplicate-key check after merges** — add a step to the merge workflow (or a pre-commit hook) that runs `yq` or a key-count grep on any modified YAML. W15 caught the bug; we shouldn't rely on observe-and-react.
- [ ] **Push `feat/content-hub-schema-sync` → main** when ready. The branch now has all today's work; it's the canonical integration. Travis controls the merge-to-main timing (and the corresponding GitHub origin push for prod release).

### Carryover from prior sessions (still open)

- [ ] **Scope the Alembic-centralization vault plan** (raised 2026-06-21, exhibit-A'd today again via the W13 "prod schema lags pbs-api" finding). One-time cost: baseline migration from current prod via `alembic --autogenerate`. Then `pbs_automation` schema flows through migrations like `pbs_garden` already does. Init-SQL files in wordpress-install shrink to `CREATE DATABASE` + `CREATE USER`. The 2026-06-21 session note has the full pitch; needs Travis green-light to turn into a vault doc.
- [ ] **CSRF nonce on save/unsave** (Phase 2 production-readiness; a-review flagged CRITICAL).
- [ ] **member-garden migrations not automated.**
- [ ] **`X-Forwarded-Proto` (ProxyFix)** in member-garden Flask.
- [ ] **WP logout confirmation nonce** (mu-plugin route for seamless logout).
- [ ] **WPCode save-button browser test** — now achievable because the sandbox has real recipe pages with WPRM markup. This is a paste-the-snippet-and-click manual test Travis can do directly; the sources are at `wpcode-snippets/save-button/`.
- [ ] **Brand color + Sunnie SVG assets** for save button + Garden (Jenny inputs).
- [ ] **pbs-video-manager `conftest.py` reconciliation** (Phase 8 fixtures lost in the same synthetic revert that hit `models.py`; W10 noted; not blocking pbs-api).
- [ ] **`init_db()` multi-worker race** in pbs-api gunicorn mode.
- [ ] **pbs-video-manager GitHub push** — main + gitea/main + gitea/master at `429383d`. GitHub stays behind until Travis is ready for prod release.

### Phase 3c candidate (next session, if Travis directs)

Saved-recipes detail page at `/the-garden/saved` (currently the Phase 3a stub). Lists saved recipes with thumbnails (joining `pbs_garden.user_interactions` + `wordpress.wp_posts`), save dates, unsave action, click-through to the real WP recipe. The sandbox now has REAL recipe data + REAL uploads, so Phase 3c can render against the real shape rather than fixtures.

## Decisions worth keeping

- **Drop + reapply over upsert for restoring third-party DBs.** Trying to merge a prod dump into an existing schema with `IF NOT EXISTS` + `INSERT IGNORE` would leave partial fixtures intermixed with real data. Clean wipe → load → verify is faster and less surprising.
- **MYSQL_PWD env var inside the container, never `-p` on the command line.** The mysql client recognizes MYSQL_PWD; passes the password without exposing it in shell history, process listings, or ansible's argument logging.
- **`--partial --inplace --progress` is the rsync triple for big-file transfers.** Resume-on-failure, no temp-then-rename, real-time progress. The right tool when scp gets flaky on large transfers.
- **Forward-fix and merge-preserve over history-rewrite.** Same posture as last session's pbs-video-manager fix. The merge commit `34c1034` keeps both parents intact; `feat/save-button-phase2` lives on as merged-but-discoverable history.
- **Pre-spawn check for "missing file" claims:** `git log --all --oneline -- <path>` before assuming. Burned three workers this week on the same lineage trap; saved memory feedback.