mcp: session-notes — Tailscale .ts.net access to Traefik — remote DNS + static cert

Sources/Homelab/2026-06-07-tailscale-tsnet-traefik-access.md

@@ -0,0 +1,63 @@
+---
+created: '2026-06-07'
+path: Sources/Homelab
+project: traefik-deployment
+tags:
+- tailscale
+- traefik
+- tls
+- dns
+- knot
+type: session-notes
+---
+
+## Outcome
+
+Laptop (`truckslaptop`) can now reach the Traefik dashboard remotely over the tailnet at `https://traefik.taila7f44e.ts.net/dashboard/` with a valid cert and clean padlock — HTTP 200 confirmed. This is a parallel access path that does not touch the existing on-LAN `herbylab.dev` + wildcard-cert setup.
+
+Access model is two-path, toggled by hand:
+- **On LAN:** `tailscale set --accept-dns=false` → normal resolver → Pi-hole → Knot → `*.herbylab.dev` names, wildcard cert. Unchanged.
+- **Remote:** `tailscale set --accept-dns=true` on the laptop → MagicDNS resolves `.ts.net` names → reach services by their Tailscale node name.
+
+## Topics Covered
+
+- Why MagicDNS short names worked via `tailscale ip` but `.ts.net` FQDNs didn't resolve in `curl`/system resolver.
+- The cert and `Host()`-routing implications of using `.ts.net` names instead of `herbylab.dev`.
+- Generating a static Tailscale cert on the Traefik box and wiring it into the file provider alongside the Cloudflare DNS-01 auto-generated wildcard.
+- Decision to keep subnet routing / split-horizon Knot off the table; accept two naming schemes by location.
+
+## Key Learnings
+
+- **MagicDNS needs a global nameserver to recurse.** With the tailnet's Global Nameservers empty, MagicDNS (`100.100.100.100` / `fd7a:115c:a1e0::53`) answers authoritatively for the tailnet but returns SERVFAIL ("recursion requested but not available") for public names. This broke `tailscale cert`'s reach to Let's Encrypt when `--accept-dns=true` was set on the Traefik box. Fix: either set `--accept-dns=false` so the box uses its own recursing resolver, or add a Global Nameserver (e.g. `1.1.1.1` or Pi-hole's tailnet IP) in the admin console. The cert command does not need MagicDNS — it only needs the box to reach Let's Encrypt over normal DNS.
+- **`tailscale set --accept-dns` is global, not per-network.** No native per-SSID switching; the two-path model is a manual toggle (acceptable here, can be automated later via a NetworkManager dispatcher hook).
+- **`resolvectl` failing with "unknown unit" = systemd-resolved not running.** On the laptop, Tailscale writes `resolv.conf` directly instead. Works fine; just know that's the mechanism rather than the resolved stub.
+- **Two cert mechanisms coexist in the file provider.** The wildcard is auto-generated by the `cloudflare` resolver (`certificates: []` is empty on purpose, Traefik fetches/renews via DNS-01). The `.ts.net` cert is a static hand-generated file pair declared explicitly under `tls.certificates`. Different mechanisms, both valid, separate dynamic files.
+- **`405` to a `curl -sI` (HEAD) against `api@internal` is a success signal** — TLS matched and routing worked; the dashboard just rejects HEAD. Confirm with a GET to `/dashboard/`.
+- **The `.ts.net` route is per-service forever.** One static cert covers the box; each service needs its own `Host(`x.taila7f44e.ts.net`)` router. The trade for skipping subnet routing / split-horizon Knot.
+
+## Config landed
+
+Cert files: `/opt/traefik/dynamic/certs/traefik.taila7f44e.ts.net.{crt,key}` (in-container: `/etc/traefik/dynamic/certs/`).
+
+New dynamic file `/opt/traefik/dynamic/tailscale.yml`:
+```yaml
+tls:
+  certificates:
+    - certFile: /etc/traefik/dynamic/certs/traefik.taila7f44e.ts.net.crt
+      keyFile: /etc/traefik/dynamic/certs/traefik.taila7f44e.ts.net.key
+http:
+  routers:
+    tailscale-dashboard:
+      rule: "Host(`traefik.taila7f44e.ts.net`)"
+      service: api@internal
+      tls: {}
+      entryPoints:
+        - websecure
+```
+
+## Follow-ons
+
+- [ ] Decide whether to add a tailnet Global Nameserver (Pi-hole tailnet IP or `1.1.1.1`) so MagicDNS recurses — would let `--accept-dns=true` stay on without breaking public resolution on any box.
+- [ ] Optional: NetworkManager dispatcher script to auto-toggle `--accept-dns` on home-network connect, removing the manual flip.
+- [ ] Renewal story for the static `.ts.net` cert — `tailscale cert` does not auto-renew like the Cloudflare resolver does; needs a timer or manual refresh before the ~90-day expiry.
+- [ ] Add `.ts.net` routers for any other service wanted remotely (per-service `Host()` matchers).