Files
serverAppConfigs/serverAppConfigs-docs/Architecture/Container Network Namespaces.md
Frank 327e78b950 docs(vault): destroy the dead weight — 53 pages and the change diary
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for
payloads that no longer run. A throughput figure for a runtime nobody operates is
not a record: AI moves faster than the vault, so the number is wrong before
anyone reads it, and a wrong number in a runbook gets believed. Per the new
purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed.

Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest):
- the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM,
  the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch
  campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two
  "Running Qwen3.6 …" pages;
- the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag)
  and the multi-device/x4-pair/hybrid/offload experiment pages;
- every vLLM page — no vLLM endpoint exists on this fleet;
- `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced
  owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff,
  self-hosted app census, the TTS survey) that were obsolete by construction;
- `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the
  page that owns the thing it governs, never to a diary.

Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras
tables keep the window, payload, runtime, reason and rollback pointer; the rate
columns are blanked. The live shapes stay verbatim — those numbers are what the
deployer and `models.yml` consume.
2026-10-19 10:56:37 -07:00

6.5 KiB

type, title, description, tags, timestamp
type title description tags timestamp
reference Container Network Namespaces How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits.
networking
podman
netavark
rootless
dankclaw
2026-09-02

Container Network Namespaces

On a rootless-Podman host, "talk to a local service" is not one question — it is one question per network namespace. A container in the host network namespace sees every host listener on 127.0.0.1; a container on a bridge network does not, and cannot be made to by name or by loopback. This page is the address map for Dankclaw (192.168.0.143) and the rule that explains it.

The rule

A container only reaches services that are reachable from inside its own network namespace:

  • Network=host → the container is in the host netns. It dials other host services on loopback (127.0.0.1:PORT) and binds any free port itself.
  • Network=<name>.network / Network=dankclaw_default (bridge, netavark) → the container has its own netns. Loopback inside it is its own loopback. The only host address it can dial is the network's gateway IP, and on a rootless host that path is served by rootlessport, which forwards published port maps only. An unpublished host listener is refused immediately (RST), not routed.
  • Container DNS names (gitea-server, authentik-postgres, valkey) resolve only inside the network that owns them, so a container on another network can never reach a peer by name — see the naming quirk below.

Live address map (Dankclaw, verified 2026-09-02)

Container NetworkMode Reachable from it
systemd-hermes (gateway) host everything on the host netns, incl. signal-cli at 127.0.0.1:8080 and the webui on :8787
hermes-webui host same; binds 0.0.0.0:8787, so it is LAN-reachable, not loopback-only
chrome-devtools-protocol host binds 127.0.0.1:9222 — deliberately loopback-only, unreachable from the LAN and from bridge nets
hermes-dashboard dankclaw_default (bridge) only published ports, via the gateway IP 10.89.1.1

Network= values are read from the live Quadlet sources in /home/dank/.config/containers/systemd/*.container; runtime network names on the host at the same date were podman, dankclaw_default, systemd-gitea systemd-searxng, systemd-open-webui.

Two consequences that are easy to miss:

  1. The Hermes gateway can dial the Signal daemon on loopback only because it runs in the host netns. The dashboard — same image, same /data — cannot; it shares state through the bind mount instead, never over the network.
  2. Caddy sits on dankclaw_default, so every upstream it proxies must be reachable from that network by IP + published port. Cross-network traffic by container DNS name is not available to it: gitea-server lives on systemd-gitea, valkey on systemd-searxng, and neither resolves from Caddy's namespace. That is why Gitea is fronted as IP:3000 and SearXNG as :8081 rather than by service name.

Naming quirk: <name>.network becomes systemd-<name>

A Quadlet Network=gitea.network yields a runtime network named systemd-gitea, not gitea. Probe with podman network ls, never by the filename, or health checks and Network-Alias plans target a network that does not exist.

Verification recipe (read-only)

# which containers are on which network? (`podman network inspect` in this
# podman version has no `.containers` field, so list from the container side)
podman ps -a --format '{{.Names}} {{.Networks}}'   # empty Networks = host or shared netns
# what listens on the host netns, and who owns it?
sudo ss -ltnp 'sport = :8080'

# what can a bridge-net container actually dial? run the probe inside its netns
podman exec <name> cat /proc/net/route          # default route → gateway IP
podman exec <name> sh -c 'timeout 2 bash -c "exec 3<>/dev/tcp/10.89.1.1/8080"; echo rc=$?'

Verified outcomes of that recipe on 2026-09-02: from inside hermes-dashboard 10.89.1.1:3000 (Gitea, published), :8081 (SearXNG via Gluetun's netns, published) and :9119 (its own published port) connect; :8080 and :8787 are refused instantly even though both listeners are bound to 0.0.0.0 in the host netns. Probes to host.containers.internal (169.254.1.2) from that network time out rather than refuse — the alias is an address on the default podman network, and this container is not on it.

Operational rules

  • Health checks for a bridge-network service must target a published port (or run via podman exec inside the unit), never localhost:<port> from the host — and never the reverse either: the deployer's health probe runs on the host, so it sees host-netns listeners that the container itself cannot dial.
  • To give a bridge-net container access to a host-netns listener, the options are (a) move it to Network=host, or (b) publish the needed port. Option (b) makes that port reachable from every netns and from the LAN — an exposure change that needs operator approval before deployment, not a silent fix.
  • chrome-devtools-protocol is intentionally Network=host with a loopback-only bind so the browser tool stays unreachable from the LAN; do not "tidy" it onto a bridge network — its two consumers (gateway, webui) are host-netns and reach it on loopback.

Exposure note: signal-cli-daemon is a host process, not a container

signal-cli-daemon.service runs as a user unit for dank (systemctl --user cat signal-cli-daemon.service) with org.asamk.signal.Main --account <redacted> daemon --http 0.0.0.0:8080. The bind is all-interfaces, so the daemon's HTTP API is reachable from anywhere on the LAN; verified from a second LAN machine with an unauthenticated GET /v1/version → HTTP 404 and body <h1>404 Not Found</h1>No context found for request (Micronaut's no-route response), i.e. the socket answers to strangers. The Hermes gateway in the host netns dials it on loopback, which is the only reason a non-loopback bind is not required by any consumer here — see Hermes for the consumer side.

See Also

  • Hermes — the gateway + Signal daemon this page's address map is keyed on
  • Hermes-Dashboard — the one Hermes container that is not in the host netns
  • Hermes-WebUI — host-netns service with a 0.0.0.0 bind and password-only auth
  • Network Architecture — the Caddy/Anubis/Authentik front door this map feeds