Files

T

Frank 69739237c8 docs(open-notebook): record the deployed shape and what the first render proved

The page described an undeployed service whose scriptwriter was going to be
frank-zealot. Both halves are now false, and the reason is worth more than the
correction.

- Status, ports, coordinates: deployed 2026-10-19, guarded vhost, API published
  on loopback only. The first episode is the proof: 747 s of speech in 301 s of
  wall clock (2.5x real time, knobs at 1), download route 200 with the bearer
  and 401 without.
- New section for the guard chain: provider-before-route, one upstream because
  the image rewrites /api/* itself, and the API_URL law — autodetect appends
  :5055, which is seerr's port on this host, so an unset API_URL yields a UI
  that loads and an app that cannot call itself.
- Provisioned records table (credentials, model rows, dank-duo, dank-daily) and
  the two contract traps measured on the live API: generate takes profile
  NAMES, and the fleet has no /v1/embeddings, so every note save fails its
  embed_note command while renders from content work fine.
- Why frank-archon writes the scripts: frank-zealot is reachable from this
  netns only at 169.254.1.2, and open-notebook's URL guard refuses to store a
  link-local base URL with no override. Census lands on [[Container Network
  Namespaces]], which also gains the root-vs-rootless publish rule — a
  root-published port is invisible to a rootless container, so "publish the
  port" was the wrong remedy as written there.
- Frank TTS: the request's model field is ignored; only voice selects a pack.

Tests: ./tests/run.sh, 455 passed.

2026-10-19 16:22:38 -07:00

8.6 KiB

Raw Blame History

type, title, description, tags, timestamp

type

title

description

Container Network Namespaces

On a rootless-Podman host, "talk to a local service" is not one question — it is one question per network namespace. A container in the host network namespace sees every host listener on 127.0.0.1; a container on a bridge network does not, and cannot be made to by name or by loopback. This page is the address map for Dankclaw (192.168.0.143) and the rule that explains it.

The rule

A container only reaches services that are reachable from inside its own network namespace:

Network=host → the container is in the host netns. It dials other host services on loopback (127.0.0.1:PORT) and binds any free port itself.
Network=<name>.network / Network=dankclaw_default (bridge, netavark) → the container has its own netns. Loopback inside it is its own loopback. The only host address it can dial is the network's gateway IP, and on a rootless host that path is served by rootlessport, which forwards published port maps only. An unpublished host listener is refused immediately (RST), not routed.
Container DNS names (gitea-server, authentik-postgres, valkey) resolve only inside the network that owns them, so a container on another network can never reach a peer by name — see the naming quirk below.

Live address map (Dankclaw, verified 2026-09-02)

Container	`NetworkMode`	Reachable from it
`systemd-hermes` (gateway)	`host`	everything on the host netns, incl. signal-cli at `127.0.0.1:8080` and the webui on `:8787`
`hermes-webui`	`host`	same; binds `0.0.0.0:8787`, so it is LAN-reachable, not loopback-only
`chrome-devtools-protocol`	`host`	binds `127.0.0.1:9222` — deliberately loopback-only, unreachable from the LAN and from bridge nets
`hermes-dashboard`	`dankclaw_default` (bridge)	only published ports, via the gateway IP `10.89.1.1`

Network= values are read from the live Quadlet sources in /home/dank/.config/containers/systemd/*.container; runtime network names on the host at the same date were podman, dankclaw_default, systemd-gitea systemd-searxng, systemd-open-webui.

Two consequences that are easy to miss:

The Hermes gateway can dial the Signal daemon on loopback only because it runs in the host netns. The dashboard — same image, same /data — cannot; it shares state through the bind mount instead, never over the network.
Caddy sits on dankclaw_default, so every upstream it proxies must be reachable from that network by IP + published port. Cross-network traffic by container DNS name is not available to it: gitea-server lives on systemd-gitea, valkey on systemd-searxng, and neither resolves from Caddy's namespace. That is why Gitea is fronted as IP:3000 and SearXNG as :8081 rather than by service name.

Naming quirk: `<name>.network` becomes `systemd-<name>`

A Quadlet Network=gitea.network yields a runtime network named systemd-gitea, not gitea. Probe with podman network ls, never by the filename, or health checks and Network-Alias plans target a network that does not exist.

Verification recipe (read-only)

# which containers are on which network? (`podman network inspect` in this
# podman version has no `.containers` field, so list from the container side)
podman ps -a --format '{{.Names}} {{.Networks}}'   # empty Networks = host or shared netns
# what listens on the host netns, and who owns it?
sudo ss -ltnp 'sport = :8080'

# what can a bridge-net container actually dial? run the probe inside its netns
podman exec <name> cat /proc/net/route          # default route → gateway IP
podman exec <name> sh -c 'timeout 2 bash -c "exec 3<>/dev/tcp/10.89.1.1/8080"; echo rc=$?'

Verified outcomes of that recipe on 2026-09-02: from inside hermes-dashboard 10.89.1.1:3000 (Gitea, published), :8081 (SearXNG via Gluetun's netns, published) and :9119 (its own published port) connect; :8080 and :8787 are refused instantly even though both listeners are bound to 0.0.0.0 in the host netns. Probes to host.containers.internal (169.254.1.2) from that network time out rather than refuse — but see the 2026-10-19 re-measure below, where the same alias answers from another dankclaw_default container.

2026-10-19: which published ports a rootless container can actually dial

Probed from inside open-notebook (rootless, on dankclaw_default and the private systemd-open-notebook), each address aimed at a live listener:

Dial	Result	Mechanism
`192.168.0.143:3000` (Gitea)	200	published by a rootless container — rootless egress crosses the host netfilter where netavark installed that DNAT
`192.168.0.143:7770` (`frank-dragoon`)	refused	published by a root container. Root and rootless Podman carry separate netavark rule sets, so a rootless container's traffic never crosses root's and a root-published port is invisible to it
`192.168.0.143:7771` (`frank-zealot`)	refused	host-netns listener, unpublished — the rule above, reconfirmed
`10.89.1.1:7771`	refused	the bridge gateway address belongs to the rootlesskit netns, not to the host
`host.containers.internal:7771` (`169.254.1.2`)	200	the gateway alias is the one address that carries a rootless container to a host-netns listener. Why `hermes-dashboard` timed out on this alias in 2026-09-02 while `open-notebook` answers today is not established; the difference to check first is the second network and where the default route lands
`192.168.0.235:7770` (`frank-archon`)	200	a different machine: plain routing, no DNAT in the path

The consequence that changes designs: a rootless container's menu of local model endpoints is frank-tts (container DNS, same network) and frank-archon (LAN address on another host). Both Dankclaw endpoints are out of reach — frank-dragoon behind root's publish, frank-zealot behind the host netns — and the only address that does reach them, 169.254.1.2, is link-local, which some applications refuse to store at all (Open Notebook meets exactly that refusal).

Operational rules

Health checks for a bridge-network service must target a published port (or run via podman exec inside the unit), never localhost:<port> from the host — and never the reverse either: the deployer's health probe runs on the host, so it sees host-netns listeners that the container itself cannot dial.
To give a bridge-net container access to a host-netns listener, the options are (a) move it to Network=host, or (b) publish the needed port — but publishing is advertised only inside the Podman that owns the container: a root container's published port stays invisible to a rootless consumer (2026-10-19 table above), so the fix must be made on the same root/rootless side as the consumer. Either way it is an exposure change — a published port also answers on the LAN — and needs operator approval before deployment, not a silent fix.
chrome-devtools-protocol is intentionally Network=host with a loopback-only bind so the browser tool stays unreachable from the LAN; do not "tidy" it onto a bridge network — its two consumers (gateway, webui) are host-netns and reach it on loopback.

Exposure note: signal-cli-daemon is a host process, not a container

signal-cli-daemon.service runs as a user unit for dank (systemctl --user cat signal-cli-daemon.service) with org.asamk.signal.Main --account <redacted> daemon --http 0.0.0.0:8080. The bind is all-interfaces, so the daemon's HTTP API is reachable from anywhere on the LAN; verified from a second LAN machine with an unauthenticated GET /v1/version → HTTP 404 and body <h1>404 Not Found</h1>No context found for request (Micronaut's no-route response), i.e. the socket answers to strangers. The Hermes gateway in the host netns dials it on loopback, which is the only reason a non-loopback bind is not required by any consumer here — see Hermes for the consumer side.

8.6 KiB Raw Blame History