122 lines
9.4 KiB
Markdown
122 lines
9.4 KiB
Markdown
---
|
|
created: '2026-05-27'
|
|
path: Sources/Homelab
|
|
project: dev-srv-split
|
|
tags:
|
|
- docker
|
|
- git
|
|
- ansible
|
|
type: project-plan
|
|
---
|
|
|
|
## Goal
|
|
|
|
Eliminate the failure mode where editing a dev folder on herbys-dev mutates a running service. Establish a clean separation between **source of truth for editing** (`/opt/projects/<svc>/`) and **what's actually running** (`/srv/<svc>/`), with releases flowing through Gitea (git tags + container registry) rather than through in-place edits.
|
|
|
|
End state: every containerized project on herbys-dev follows the same shape, deploys from a frozen artifact, and can be rolled back by changing one reference. Dev work and live services are independent enough that breaking one doesn't break the other.
|
|
|
|
## Locked Decisions
|
|
|
|
- **Two directory trees on herbys-dev.** `/opt/projects/<svc>/` is the dev tree (git working copy, edits happen here). `/srv/<svc>/` is the deploy tree (what's actually running, owns runtime state).
|
|
- **Deploy tree is a pinned git checkout, not a copy.** `/srv/<svc>/` is `git clone` of the project repo, checked out at a release tag (e.g. `v0.4.0`). It is never edited in place. Upgrades are `git fetch && git checkout <new-tag>`.
|
|
- **Runtime state lives only in `/srv/`.** `data/`, real `.env` files, anything that accumulates over time. The dev tree has no live data and no real secrets.
|
|
- **`data/` and `.env` are gitignored in every source repo.** Required for the deploy `git checkout` model to work without clobbering live state or pulling wrong secrets.
|
|
- **Volume paths in compose files are relative (`./data`), not absolute.** The same compose file works in dev or deploy without modification — compose resolves `./` from the invocation directory.
|
|
- **Two compose file patterns, chosen per project (not per file in the same repo):**
|
|
- *App-style projects* (image is the deliverable): `compose.dev.yml` in source for iteration, hand-written `compose.yml` in `/srv/` referencing image by tag.
|
|
- *System-style projects* (compose is part of the deliverable, e.g. OB1): single `docker-compose.yml` in source, frozen at a git tag, deploy tree is a checkout of that tag.
|
|
- **Release artifacts live in Gitea.** Git commits and tags in the repo; built images in the Gitea container registry. The deploy tree never touches source code outside its own git checkout.
|
|
- **Migration is per-project, not bulk.** Lowest-risk service first to shake out the pattern; OB1 and other production-adjacent services come after the pattern is proven.
|
|
- **Ansible is out of scope for this plan.** This plan establishes the manual shape. Ansible automation of `/srv/` provisioning is a follow-on once the pattern is settled.
|
|
|
|
## Open Items
|
|
|
|
- [ ] Gitea Actions runner integration — runner exists on Gitea but Travis doesn't yet understand its role in this flow. Lovebug flagged it as relevant. Expected role: automate `docker build && docker push` to the Gitea registry on tag push, so releases don't require manual local builds. To be explored during implementation, not designed upfront.
|
|
- [ ] Concrete project list and migration order — Travis + Lovebug to compile separately.
|
|
- [ ] Image tag convention — semver (`v0.4.0`) vs. semver + git SHA (`v0.4.0-a4f3c91`) vs. digest pinning. Defer until first migration; whatever shakes out can be standardized after.
|
|
- [ ] Whether dev iteration on system-style projects (OB1) needs a separate dev directory at all, or whether iterating in `/opt/projects/ob1/` and only `git checkout`-ing the tag in `/srv/ob1/` is sufficient. Lean toward the simpler version; revisit if it bites.
|
|
- [ ] Cleanup of the existing dev tree once `/srv/` is established — remove `data/` and live `.env` from the dev tree's working copy, add to `.gitignore` if not already, confirm nothing else has leaked in.
|
|
|
|
## Phases
|
|
|
|
### Phase 1 — Pilot migration (lowest-risk service)
|
|
|
|
- [ ] Pick a single low-stakes service currently deploying from `/opt/projects/`.
|
|
- [ ] Verify `data/` and `.env` are gitignored in the source repo; add if missing.
|
|
- [ ] Verify volume paths in compose are relative (`./data`), not absolute.
|
|
- [ ] Tag the current state of the source repo (`v<current>`), push tag to Gitea.
|
|
- [ ] `docker compose down` in the dev tree.
|
|
- [ ] Move runtime state out: `sudo mkdir -p /srv/<svc> && sudo mv data .env /srv/<svc>/`.
|
|
- [ ] `git clone -b v<current>` the repo into `/srv/<svc>/`.
|
|
- [ ] Set ownership: `sudo chown -R $USER:devprojects /srv/<svc>`.
|
|
- [ ] `docker compose up -d` from `/srv/<svc>/`; verify service is healthy.
|
|
- [ ] Validate that editing in `/opt/projects/<svc>/` no longer affects the running container.
|
|
- [ ] Document any pattern adjustments discovered during the pilot.
|
|
|
|
### Phase 2 — Pattern documentation
|
|
|
|
- [ ] Write the canonical per-project migration runbook based on what was learned in Phase 1.
|
|
- [ ] Decide and document image tag convention (resolves Open Item).
|
|
- [ ] Decide and document whether system-style projects need a separate dev directory (resolves Open Item).
|
|
- [ ] Add a section to project READMEs (or a single shared doc) describing the dev/srv split so future-Travis and Lovebug both know the contract.
|
|
|
|
### Phase 3 — Gitea runner integration (exploratory)
|
|
|
|
- [ ] Understand current Gitea Actions runner setup — where it runs, what permissions it has, what it currently does.
|
|
- [ ] Design a minimal workflow: on `git push --tags`, build the image and push to the Gitea registry.
|
|
- [ ] Pilot the runner workflow on the same low-risk service from Phase 1.
|
|
- [ ] Document the trigger pattern (tag format, registry path convention) so it generalizes.
|
|
- [ ] Decide whether the runner publishes images on every tag, every commit to `main`, or both.
|
|
|
|
### Phase 4 — Roll out to remaining services
|
|
|
|
- [ ] Apply the runbook to remaining containerized projects, ordered by risk (lowest first, production-critical last).
|
|
- [ ] OB1 explicitly called out — was the trigger for this work; expect surprises around the Supabase + Postgres + MCP composition.
|
|
- [ ] For each service: gitignore audit, relative-path audit, tag the source, move state to `/srv/`, deploy tree as pinned checkout, validate isolation.
|
|
|
|
### Phase 5 — Ansible follow-on (out of scope here, captured for continuity)
|
|
|
|
- [ ] Once the pattern is stable across services, design Ansible roles that automate `/srv/` provisioning: `git clone`/`checkout`, render `.env` from secrets, `docker compose pull && up -d`.
|
|
- [ ] Tracked separately; this plan ends at "manual pattern proven and documented."
|
|
|
|
## Notes
|
|
|
|
### Why this exists
|
|
|
|
Current pattern: `docker compose up` runs from inside `/opt/projects/<svc>/`, which is also the git working tree. Editing files (or switching branches) mutates the live service's config and state. This already caused a near-miss on OB1 — changes that would have broken the live service because it was pointed at the dev folder.
|
|
|
|
### The mental model
|
|
|
|
The unit of release is **whatever the deliverable is** for that project. For app-style projects, the deliverable is a tagged image in the registry. For system-style projects (OB1), the deliverable is a tagged commit of the repo containing the compose file, configs, and references to custom images. In both cases, "release" produces an artifact in a remote store (Gitea), and "deploy" means pointing `/srv/<svc>/` at that artifact.
|
|
|
|
### What stays still during a release
|
|
|
|
- `compose.dev.yml` (if present): unchanged. Dev convenience, not part of the release.
|
|
- `/srv/<svc>/compose.yml` or the compose file inside the deploy checkout: unchanged across most releases.
|
|
- Source tree layout: unchanged.
|
|
|
|
### What moves during a release
|
|
|
|
- App-style: a new image tag in the Gitea registry; one line in `/srv/<svc>/.env` (`IMAGE_TAG=...`).
|
|
- System-style: a new git tag on the source repo; `git checkout <new-tag>` in `/srv/<svc>/`.
|
|
|
|
### Single-host wrinkle
|
|
|
|
herbys-dev is both the dev host and the deploy host. The pattern still works — discipline replaces physical separation. Directory boundaries (`/opt/projects/` vs `/srv/`), different compose filenames where applicable, and different project names (compose derives project name from directory) keep the two contexts from colliding.
|
|
|
|
### Why the Gitea runner matters (best guess pending exploration)
|
|
|
|
Without a runner, every release requires manually running `docker build && docker push` on herbys-dev (or wherever). The runner automates that step on tag push, so releases become "tag and push" rather than "tag, push, build, push image, then deploy." It also removes herbys-dev from the build path for projects that get a runner workflow, which is a small step toward eventual dev-host / deploy-host separation if Travis ever splits them.
|
|
|
|
### Failure modes to watch during migration
|
|
|
|
- Absolute volume paths in compose files breaking when the file relocates to `/srv/`.
|
|
- Live `.env` or `data/` accidentally committed to the source repo before gitignore is added.
|
|
- Compose project name collisions if dev and deploy versions of the same service are running simultaneously (compose derives project name from directory, so `/opt/projects/foo/` and `/srv/foo/` get the same name).
|
|
- Custom images referenced in compose files not actually existing in the Gitea registry at the tag the deploy tree is checked out at (registry and git tag drift).
|
|
|
|
### Related
|
|
|
|
- Lovebug suggested incorporating the Gitea Actions runner; runner role to be learned during Phase 3.
|
|
- Ansible deployment work is a downstream consumer of this pattern, not a prerequisite.
|
|
- Backups: `/srv/<svc>/data/` becomes the canonical location for service state; backup tooling should target `/srv/` rather than the dev tree. |