---
created: '2026-06-08'
path: Sources/Homelab
project: petal-dispatch-p4
tags:
- tailscale
- split-dns
- knot
- subnet-router
- petal-dispatch
- remote-access
- homelab
- p4
type: session-notes
---

## Outcome

Off-LAN remote access to `*.herbylab.dev` over Tailscale is working end to end. The petal-dispatch Android app and web client both connect to `https://dispatch.herbylab.dev/` from off-LAN (verified on hotel wifi), with SSE live and session context (`session_uuid`) carrying across devices. No app changes required — the hardcoded URL just works. This was the prerequisite for the phone app off-LAN, answering the open question of whether remote access plumbing was needed (it was).

Chose **Path A** (split-DNS so original URLs work everywhere) over Path B (`.ts.net` MagicDNS hostname). A is the permanent answer; B was a lighter-touch stopgap. Committed to A and landed it.

## Working configuration

**1. Tailscale on knot (LXC 103, unprivileged)**
- Required `/dev/net/tun` passthrough in `/etc/pve/lxc/103.conf` (on PVE host):
```
lxc.cgroup2.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file
```
- `pct restart 103`, then install via the standard script, then `tailscale up`.

**2. knot listens on its tailnet IP**
- Added a third listen line in `/etc/knot/knot.conf` server block alongside the LAN and loopback addresses:
```
listen: <knot-tailnet-100.x>@53
```
- This is config-file-based, so it is **persistent** across knot restarts and reboots — not a runtime setting that needs re-applying.
- knot requires full `systemctl restart knot` to re-bind sockets (`knotc reload` does not).

**3. Tailscale admin DNS (split DNS)**
- Restricted/custom nameserver → knot's tailnet IP, restricted to domain `herbylab.dev`.
- "Override local DNS" on.

**4. Subnet router (because knot returns LAN IPs)**
- knot returns `10.0.11.20` (Traefik VM LAN IP), which tailnet clients can't reach directly.
- herbys-dev advertises the LAN: `sudo tailscale up --advertise-routes=10.0.0.0/16` (`/16` covers both 10.0.11.x and 10.0.21.x).
- Route **must be approved in the Tailscale admin machines page** — this is a separate gate from advertising.
- Clients accept with `sudo tailscale up --accept-routes`.

## Key learnings

- **`tailscale up` is all-or-nothing on prefs.** A bare `tailscale up` or a `systemctl restart tailscaled` silently drops any flag not repeated — `--advertise-routes` on the subnet router, `--accept-routes` on the client. Always repeat the full flag set. This was the single biggest time sink: the architecture was correct the whole time; advertised routes had been silently cleared by intervening bare restarts.
- **Route approval is a distinct step from route advertising.** A node can advertise a subnet and still show empty `AllowedIPs` until the route is approved in admin. No amount of `--accept-routes` on the client pulls a route that isn't authorized server-side. Verify with `tailscale status --json | grep -A20 <node>` → look for the subnet in `AllowedIPs`.
- **DNS working ≠ routing working.** Split-DNS resolving correctly (`dig` returns the right IP via `100.100.100.100`) is independent of whether the returned IP is reachable. The DNS-works/ping-fails split is the expected signature of "LAN IP returned, subnet router/approval not yet complete," not a bug.
- **Diagnosis order that works:** confirm client connected (`tailscale status`) → confirm route present in client table (`ip route | grep tailscale`) → if absent, confirm the route is being advertised AND approved (`AllowedIPs` on the advertising node) before touching the client again.

## Reachability note (deferred, not done)

This path uses herbys-dev as subnet router, which means inter-VLAN forwarding (herbys-dev is on a different VLAN than 10.0.11.20). It works today. A cleaner future option exists: since the Traefik VM is itself a tailnet node (100.86.14.2), pointing knot's service records at Traefik's tailnet IP would eliminate the subnet router, IP forwarding, and the VLAN hop entirely (Path B-style records, no MagicDNS dependency). Not needed now; noted for if the subnet-router path ever gets fragile.

## Follow-up trigger

- Infra node Tailscale keys are on the default **~6-month expiry** (herbys-dev, knot, traefik). When they expire, remote access dies silently until re-auth. Decided to leave as-is for now rather than disable expiry. **Trigger: ~November 2026** — re-auth or disable key expiry before then. Breadcrumb lives here so a cold debug session finds the cause fast.