wiki-vault/Sources/Homelab/dev-srv-split.md

created	path	project	tags	type
2026-05-27	Sources/Homelab	dev-srv-split	docker git ansible	project-plan

Goal

Eliminate the failure mode where editing a dev folder on herbys-dev mutates a running service. Establish a clean separation between source of truth for editing (/opt/projects/<svc>/) and what's actually running (/srv/<svc>/), with releases flowing through Gitea (git tags + container registry) rather than through in-place edits.

End state: every containerized project on herbys-dev follows the same shape, deploys from a frozen artifact, and can be rolled back by changing one reference. Dev work and live services are independent enough that breaking one doesn't break the other.

Locked Decisions

• Two directory trees on herbys-dev. /opt/projects/<svc>/ is the dev tree (git working copy, edits happen here). /srv/<svc>/ is the deploy tree (what's actually running, owns runtime state).
• Deploy tree is a pinned git checkout, not a copy. /srv/<svc>/ is git clone of the project repo, checked out at a release tag (e.g. v0.4.0). It is never edited in place. Upgrades are git fetch && git checkout <new-tag>.
• Runtime state lives only in /srv/. data/, real .env files, anything that accumulates over time. The dev tree has no live data and no real secrets.
• data/ and .env are gitignored in every source repo. Required for the deploy git checkout model to work without clobbering live state or pulling wrong secrets.
• Volume paths in compose files are relative (./data), not absolute. The same compose file works in dev or deploy without modification — compose resolves ./ from the invocation directory.
• Two compose file patterns, chosen per project (not per file in the same repo):
  • App-style projects (image is the deliverable): compose.dev.yml in source for iteration, hand-written compose.yml in /srv/ referencing image by tag.
  • System-style projects (compose is part of the deliverable, e.g. OB1): single docker-compose.yml in source, frozen at a git tag, deploy tree is a checkout of that tag.
• Release artifacts live in Gitea. Git commits and tags in the repo; built images in the Gitea container registry. The deploy tree never touches source code outside its own git checkout.
• Migration is per-project, not bulk. Lowest-risk service first to shake out the pattern; OB1 and other production-adjacent services come after the pattern is proven.
• Ansible is out of scope for this plan. This plan establishes the manual shape. Ansible automation of /srv/ provisioning is a follow-on once the pattern is settled.

Open Items

• Gitea Actions runner integration — runner exists on Gitea but Travis doesn't yet understand its role in this flow. Lovebug flagged it as relevant. Expected role: automate docker build && docker push to the Gitea registry on tag push, so releases don't require manual local builds. To be explored during implementation, not designed upfront.
• Concrete project list and migration order — Travis + Lovebug to compile separately.
• Image tag convention — semver (v0.4.0) vs. semver + git SHA (v0.4.0-a4f3c91) vs. digest pinning. Defer until first migration; whatever shakes out can be standardized after.
• Whether dev iteration on system-style projects (OB1) needs a separate dev directory at all, or whether iterating in /opt/projects/ob1/ and only git checkout-ing the tag in /srv/ob1/ is sufficient. Lean toward the simpler version; revisit if it bites.
• Cleanup of the existing dev tree once /srv/ is established — remove data/ and live .env from the dev tree's working copy, add to .gitignore if not already, confirm nothing else has leaked in.

Phases

Phase 1 — Pilot migration (lowest-risk service)

• Pick a single low-stakes service currently deploying from /opt/projects/.
• Verify data/ and .env are gitignored in the source repo; add if missing.
• Verify volume paths in compose are relative (./data), not absolute.
• Tag the current state of the source repo (v<current>), push tag to Gitea.
• docker compose down in the dev tree.
• Move runtime state out: sudo mkdir -p /srv/<svc> && sudo mv data .env /srv/<svc>/.
• git clone -b v<current> the repo into /srv/<svc>/.
• Set ownership: sudo chown -R $USER:devprojects /srv/<svc>.
• docker compose up -d from /srv/<svc>/; verify service is healthy.
• Validate that editing in /opt/projects/<svc>/ no longer affects the running container.
• Document any pattern adjustments discovered during the pilot.

Phase 2 — Pattern documentation

• Write the canonical per-project migration runbook based on what was learned in Phase 1.
• Decide and document image tag convention (resolves Open Item).
• Decide and document whether system-style projects need a separate dev directory (resolves Open Item).
• Add a section to project READMEs (or a single shared doc) describing the dev/srv split so future-Travis and Lovebug both know the contract.

Phase 3 — Gitea runner integration (exploratory)

• Understand current Gitea Actions runner setup — where it runs, what permissions it has, what it currently does.
• Design a minimal workflow: on git push --tags, build the image and push to the Gitea registry.
• Pilot the runner workflow on the same low-risk service from Phase 1.
• Document the trigger pattern (tag format, registry path convention) so it generalizes.
• Decide whether the runner publishes images on every tag, every commit to main, or both.

Phase 4 — Roll out to remaining services

• Apply the runbook to remaining containerized projects, ordered by risk (lowest first, production-critical last).
• OB1 explicitly called out — was the trigger for this work; expect surprises around the Supabase + Postgres + MCP composition.
• For each service: gitignore audit, relative-path audit, tag the source, move state to /srv/, deploy tree as pinned checkout, validate isolation.

Phase 5 — Ansible follow-on (out of scope here, captured for continuity)

• Once the pattern is stable across services, design Ansible roles that automate /srv/ provisioning: git clone/checkout, render .env from secrets, docker compose pull && up -d.
• Tracked separately; this plan ends at "manual pattern proven and documented."

Notes

Why this exists

Current pattern: docker compose up runs from inside /opt/projects/<svc>/, which is also the git working tree. Editing files (or switching branches) mutates the live service's config and state. This already caused a near-miss on OB1 — changes that would have broken the live service because it was pointed at the dev folder.

The mental model

The unit of release is whatever the deliverable is for that project. For app-style projects, the deliverable is a tagged image in the registry. For system-style projects (OB1), the deliverable is a tagged commit of the repo containing the compose file, configs, and references to custom images. In both cases, "release" produces an artifact in a remote store (Gitea), and "deploy" means pointing /srv/<svc>/ at that artifact.

What stays still during a release

• compose.dev.yml (if present): unchanged. Dev convenience, not part of the release.
• /srv/<svc>/compose.yml or the compose file inside the deploy checkout: unchanged across most releases.
• Source tree layout: unchanged.

What moves during a release

• App-style: a new image tag in the Gitea registry; one line in /srv/<svc>/.env (IMAGE_TAG=...).
• System-style: a new git tag on the source repo; git checkout <new-tag> in /srv/<svc>/.

Single-host wrinkle

herbys-dev is both the dev host and the deploy host. The pattern still works — discipline replaces physical separation. Directory boundaries (/opt/projects/ vs /srv/), different compose filenames where applicable, and different project names (compose derives project name from directory) keep the two contexts from colliding.

Why the Gitea runner matters (best guess pending exploration)

Without a runner, every release requires manually running docker build && docker push on herbys-dev (or wherever). The runner automates that step on tag push, so releases become "tag and push" rather than "tag, push, build, push image, then deploy." It also removes herbys-dev from the build path for projects that get a runner workflow, which is a small step toward eventual dev-host / deploy-host separation if Travis ever splits them.

Failure modes to watch during migration

• Absolute volume paths in compose files breaking when the file relocates to /srv/.
• Live .env or data/ accidentally committed to the source repo before gitignore is added.
• Compose project name collisions if dev and deploy versions of the same service are running simultaneously (compose derives project name from directory, so /opt/projects/foo/ and /srv/foo/ get the same name).
• Custom images referenced in compose files not actually existing in the Gitea registry at the tag the deploy tree is checked out at (registry and git tag drift).

Related

• Lovebug suggested incorporating the Gitea Actions runner; runner role to be learned during Phase 3.
• Ansible deployment work is a downstream consumer of this pattern, not a prerequisite.
• Backups: /srv/<svc>/data/ becomes the canonical location for service state; backup tooling should target /srv/ rather than the dev tree.