diff --git a/Sources/Homelab/dev-srv-split.md b/Sources/Homelab/dev-srv-split.md new file mode 100644 index 0000000..ba01dc8 --- /dev/null +++ b/Sources/Homelab/dev-srv-split.md @@ -0,0 +1,122 @@ +--- +created: '2026-05-27' +path: Sources/Homelab +project: dev-srv-split +tags: +- docker +- git +- ansible +type: project-plan +--- + +## Goal + +Eliminate the failure mode where editing a dev folder on herbys-dev mutates a running service. Establish a clean separation between **source of truth for editing** (`/opt/projects//`) and **what's actually running** (`/srv//`), with releases flowing through Gitea (git tags + container registry) rather than through in-place edits. + +End state: every containerized project on herbys-dev follows the same shape, deploys from a frozen artifact, and can be rolled back by changing one reference. Dev work and live services are independent enough that breaking one doesn't break the other. + +## Locked Decisions + +- **Two directory trees on herbys-dev.** `/opt/projects//` is the dev tree (git working copy, edits happen here). `/srv//` is the deploy tree (what's actually running, owns runtime state). +- **Deploy tree is a pinned git checkout, not a copy.** `/srv//` is `git clone` of the project repo, checked out at a release tag (e.g. `v0.4.0`). It is never edited in place. Upgrades are `git fetch && git checkout `. +- **Runtime state lives only in `/srv/`.** `data/`, real `.env` files, anything that accumulates over time. The dev tree has no live data and no real secrets. +- **`data/` and `.env` are gitignored in every source repo.** Required for the deploy `git checkout` model to work without clobbering live state or pulling wrong secrets. +- **Volume paths in compose files are relative (`./data`), not absolute.** The same compose file works in dev or deploy without modification — compose resolves `./` from the invocation directory. +- **Two compose file patterns, chosen per project (not per file in the same repo):** + - *App-style projects* (image is the deliverable): `compose.dev.yml` in source for iteration, hand-written `compose.yml` in `/srv/` referencing image by tag. + - *System-style projects* (compose is part of the deliverable, e.g. OB1): single `docker-compose.yml` in source, frozen at a git tag, deploy tree is a checkout of that tag. +- **Release artifacts live in Gitea.** Git commits and tags in the repo; built images in the Gitea container registry. The deploy tree never touches source code outside its own git checkout. +- **Migration is per-project, not bulk.** Lowest-risk service first to shake out the pattern; OB1 and other production-adjacent services come after the pattern is proven. +- **Ansible is out of scope for this plan.** This plan establishes the manual shape. Ansible automation of `/srv/` provisioning is a follow-on once the pattern is settled. + +## Open Items + +- [ ] Gitea Actions runner integration — runner exists on Gitea but Travis doesn't yet understand its role in this flow. Lovebug flagged it as relevant. Expected role: automate `docker build && docker push` to the Gitea registry on tag push, so releases don't require manual local builds. To be explored during implementation, not designed upfront. +- [ ] Concrete project list and migration order — Travis + Lovebug to compile separately. +- [ ] Image tag convention — semver (`v0.4.0`) vs. semver + git SHA (`v0.4.0-a4f3c91`) vs. digest pinning. Defer until first migration; whatever shakes out can be standardized after. +- [ ] Whether dev iteration on system-style projects (OB1) needs a separate dev directory at all, or whether iterating in `/opt/projects/ob1/` and only `git checkout`-ing the tag in `/srv/ob1/` is sufficient. Lean toward the simpler version; revisit if it bites. +- [ ] Cleanup of the existing dev tree once `/srv/` is established — remove `data/` and live `.env` from the dev tree's working copy, add to `.gitignore` if not already, confirm nothing else has leaked in. + +## Phases + +### Phase 1 — Pilot migration (lowest-risk service) + +- [ ] Pick a single low-stakes service currently deploying from `/opt/projects/`. +- [ ] Verify `data/` and `.env` are gitignored in the source repo; add if missing. +- [ ] Verify volume paths in compose are relative (`./data`), not absolute. +- [ ] Tag the current state of the source repo (`v`), push tag to Gitea. +- [ ] `docker compose down` in the dev tree. +- [ ] Move runtime state out: `sudo mkdir -p /srv/ && sudo mv data .env /srv//`. +- [ ] `git clone -b v` the repo into `/srv//`. +- [ ] Set ownership: `sudo chown -R $USER:devprojects /srv/`. +- [ ] `docker compose up -d` from `/srv//`; verify service is healthy. +- [ ] Validate that editing in `/opt/projects//` no longer affects the running container. +- [ ] Document any pattern adjustments discovered during the pilot. + +### Phase 2 — Pattern documentation + +- [ ] Write the canonical per-project migration runbook based on what was learned in Phase 1. +- [ ] Decide and document image tag convention (resolves Open Item). +- [ ] Decide and document whether system-style projects need a separate dev directory (resolves Open Item). +- [ ] Add a section to project READMEs (or a single shared doc) describing the dev/srv split so future-Travis and Lovebug both know the contract. + +### Phase 3 — Gitea runner integration (exploratory) + +- [ ] Understand current Gitea Actions runner setup — where it runs, what permissions it has, what it currently does. +- [ ] Design a minimal workflow: on `git push --tags`, build the image and push to the Gitea registry. +- [ ] Pilot the runner workflow on the same low-risk service from Phase 1. +- [ ] Document the trigger pattern (tag format, registry path convention) so it generalizes. +- [ ] Decide whether the runner publishes images on every tag, every commit to `main`, or both. + +### Phase 4 — Roll out to remaining services + +- [ ] Apply the runbook to remaining containerized projects, ordered by risk (lowest first, production-critical last). +- [ ] OB1 explicitly called out — was the trigger for this work; expect surprises around the Supabase + Postgres + MCP composition. +- [ ] For each service: gitignore audit, relative-path audit, tag the source, move state to `/srv/`, deploy tree as pinned checkout, validate isolation. + +### Phase 5 — Ansible follow-on (out of scope here, captured for continuity) + +- [ ] Once the pattern is stable across services, design Ansible roles that automate `/srv/` provisioning: `git clone`/`checkout`, render `.env` from secrets, `docker compose pull && up -d`. +- [ ] Tracked separately; this plan ends at "manual pattern proven and documented." + +## Notes + +### Why this exists + +Current pattern: `docker compose up` runs from inside `/opt/projects//`, which is also the git working tree. Editing files (or switching branches) mutates the live service's config and state. This already caused a near-miss on OB1 — changes that would have broken the live service because it was pointed at the dev folder. + +### The mental model + +The unit of release is **whatever the deliverable is** for that project. For app-style projects, the deliverable is a tagged image in the registry. For system-style projects (OB1), the deliverable is a tagged commit of the repo containing the compose file, configs, and references to custom images. In both cases, "release" produces an artifact in a remote store (Gitea), and "deploy" means pointing `/srv//` at that artifact. + +### What stays still during a release + +- `compose.dev.yml` (if present): unchanged. Dev convenience, not part of the release. +- `/srv//compose.yml` or the compose file inside the deploy checkout: unchanged across most releases. +- Source tree layout: unchanged. + +### What moves during a release + +- App-style: a new image tag in the Gitea registry; one line in `/srv//.env` (`IMAGE_TAG=...`). +- System-style: a new git tag on the source repo; `git checkout ` in `/srv//`. + +### Single-host wrinkle + +herbys-dev is both the dev host and the deploy host. The pattern still works — discipline replaces physical separation. Directory boundaries (`/opt/projects/` vs `/srv/`), different compose filenames where applicable, and different project names (compose derives project name from directory) keep the two contexts from colliding. + +### Why the Gitea runner matters (best guess pending exploration) + +Without a runner, every release requires manually running `docker build && docker push` on herbys-dev (or wherever). The runner automates that step on tag push, so releases become "tag and push" rather than "tag, push, build, push image, then deploy." It also removes herbys-dev from the build path for projects that get a runner workflow, which is a small step toward eventual dev-host / deploy-host separation if Travis ever splits them. + +### Failure modes to watch during migration + +- Absolute volume paths in compose files breaking when the file relocates to `/srv/`. +- Live `.env` or `data/` accidentally committed to the source repo before gitignore is added. +- Compose project name collisions if dev and deploy versions of the same service are running simultaneously (compose derives project name from directory, so `/opt/projects/foo/` and `/srv/foo/` get the same name). +- Custom images referenced in compose files not actually existing in the Gitea registry at the tag the deploy tree is checked out at (registry and git tag drift). + +### Related + +- Lovebug suggested incorporating the Gitea Actions runner; runner role to be learned during Phase 3. +- Ansible deployment work is a downstream consumer of this pattern, not a prerequisite. +- Backups: `/srv//data/` becomes the canonical location for service state; backup tooling should target `/srv/` rather than the dev tree. \ No newline at end of file