Compare commits

...
2 Commits
Author SHA1 Message Date
Frank e914ecbca6 docs(pi-hole): the record-load trap rides a container restart, not a boot
The 2026-10-19 write-up inferred a host restart from FTL's 16:25 start line. It was wrong: uptime -s reads 2026-10-18 01:24 (two days up) and the user journal shows 'Stopping pihole.service' at 16:25:23 with StartedAt 16:25:27 — a plain container restart inside the open-notebook deploy window. That moves where the verification belongs: after every pihole restart, not just after a reboot.
2026-10-20 10:56:39 -07:00
Frank 1eebf3b129 docs(netns): the by-name path to frank-dragoon is verified from the consumer
open-notebook resolves frank-dragoon-halogen to 10.89.1.39, GET :8731/v1/models answers, and the engine logs the app's POST /v1/embeddings 200. That makes the 2026-10-19 dial census a pre-move record: pasta's forwarding rule is unchanged, the endpoint was moved out of pasta, so the 'menu is frank-tts + frank-archon' line is now past tense. Frontmatter date was still 2026-09-02 against October content; bumped.
2026-10-19 22:48:44 -07:00
2 changed files with 13 additions and 4 deletions
@@ -8,7 +8,7 @@ tags:
- netavark
- rootless
- dankclaw
timestamp: 2026-09-02
timestamp: 2026-10-19
---
# Container Network Namespaces
@@ -106,8 +106,8 @@ private `systemd-open-notebook`), each address aimed at a live listener:
| `host.containers.internal:7771` (`169.254.1.2`) | **200** | the gateway alias is the one address that carries a rootless container to a host-netns listener. Why `hermes-dashboard` timed out on this alias in 2026-09-02 while `open-notebook` answers today is **not established**; the difference to check first is the second network and where the default route lands |
| `192.168.0.235:7770` (`frank-archon`) | **200** | a different machine: plain routing, no DNAT in the path |
The consequence that changes designs: **a rootless container's menu of local model
endpoints is `frank-tts` (container DNS, same network) and `frank-archon` (LAN
The consequence that changed designs: **a rootless container's menu of local model
endpoints was only `frank-tts` (container DNS, same network) and `frank-archon` (LAN
address on another host).** A `pasta` endpoint has no address an app container can
use: the LAN address is refused, and the one alias that answers is link-local,
which some applications refuse to store at all ([[Open Notebook]] meets exactly
@@ -127,6 +127,15 @@ API port becomes name-resolvable, which is the same trust the published 7770
already carries on the LAN. `frank-zealot` (7771) stays a host-netns binary and
remains unreachable except through the link-local alias.
The move is verified from the consumer side, so the table above reads as the
**pre-move** census and not as today's menu: inside `open-notebook`,
`getent hosts frank-dragoon-halogen` → `10.89.1.39`, `GET :8731/v1/models`
lists both served models, and the engine's log carries the app's
`POST /v1/embeddings 200` from that address (2026-10-19) — see
[[Open Notebook]] for what runs over it. What the table still owns is the
pasta rule itself: nothing about pasta's forwarding changed, the endpoint was
moved out of it.
## Operational rules
- Health checks for a bridge-network service must target a **published** port (or
@@ -46,7 +46,7 @@ The file format is hosts-style (`<IP> <name> [<name> ...]`), one A record per Ca
**Pitfall (2026-08-24):** the first quadlet mounted the file into `/etc/dnsmasq.d/`. FTL never read it; names only resolved from the query cache. A fresh name returned NXDOMAIN. Mount into `/etc/pihole/hosts/`, not `/etc/dnsmasq.d/`.
**Pitfall (2026-10-19): FTL can come up with the record file mounted and still serve none of it.** After a Dankclaw restart every `franksdank.cloud` name went NXDOMAIN for LAN clients while **external names kept answering** — from FTL's cache, which is exactly what makes the failure read like a client or an upstream problem. The instrument is on the box: `getent hosts grafana.franksdank.cloud` (or `dig +short @192.168.0.143 <name>`) answering empty means FTL is holding no local records at all; here the file was present and readable inside the container (`-rw-rw-r-- … /etc/pihole/hosts/addn-hosts`, 44 records) and `dns.hosts` in `pihole.toml` was intact. `podman exec pihole pihole restartdns reload` had every name answering within seconds. The mechanism is **not established** — the same mounts and the same config read fine before and after the reload — so treat it as a boot-time load failure of the record sources, not as a lost or unmounted file. What it cost: the whole Playwright gate, 18 specs across every vhost failing `ERR_NAME_NOT_RESOLVED` while the fleet itself was healthy ([[E2E Suite]]).
**Pitfall (2026-10-19): FTL can come up with the record file mounted and still serve none of it.** A plain `systemctl --user restart pihole` (16:25, inside the open-notebook deploy window) left every `franksdank.cloud` name NXDOMAIN for LAN clients while **external names kept answering** — from FTL's cache, which is exactly what makes the failure read like a client or an upstream problem. **The trigger is a container restart, not a boot**, and that matters for where the check belongs: `uptime -s` reads 2026-10-18 01:24 (two days of uptime) while `journalctl --user -u pihole.service` shows `Stopping pihole.service` at 16:25:23 and the container's `StartedAt` at 16:25:27 — so verify after **any** pihole restart, deploy-triggered or manual, not only after a reboot. The instrument is on the box: `getent hosts grafana.franksdank.cloud` (or `dig +short @192.168.0.143 <name>`) answering empty means FTL is holding no local records at all; here the file was present and readable inside the container (`-rw-rw-r-- … /etc/pihole/hosts/addn-hosts`, 44 records) and `dns.hosts` in `pihole.toml` was intact. `podman exec pihole pihole restartdns reload` had every name answering within seconds. The mechanism is **not established** — the same mounts and the same config read fine before and after the reload — so treat it as a load failure of the record sources at FTL start, not as a lost or unmounted file. What it cost: the whole Playwright gate, 18 specs across every vhost failing `ERR_NAME_NOT_RESOLVED` while the fleet itself was healthy ([[E2E Suite]]).
## Upstream