--- name: podman-netavark-host-port-probe description: "Diagnose \"connection refused\" when a rootless-podman container tries to reach a host service port on this netavark host; proves the published-port-only quirk and picks the working path (host.containers.internal)." --- # Podman/netavark: container → host port connectivity probe Use when a containerized service on a rootless-podman + netavark host cannot reach a HOST process by LAN IP (connection refused immediately, not a timeout) even though `ss -ltnp` proves the listener is up in the host netns. ## Why this happens On netavark-backed networks, container→host-LAN-IP traffic is only delivered for ports that have a live port-forward/DNAT (published ports). Un-published host ports refuse instantly — there is no firewall rule and no stale DNAT to find; `nft list tables`/`iptables -t nat` will be empty. Do not hunt for REJECT rules; probe instead. ## Probe recipe (proves the quirk in one command) ```bash ssh "podman run --rm --network --entrypoint sh docker.io/library/busybox:latest -c 'for p in ; do nc -w2 -z -v \$p >/dev/null 2>&1 && echo port-\$p-OK || echo port-\$p-BAD; done'" ``` - Published port OK + target port BAD ⇒ published-port-only quirk confirmed. - Also probe the alias path from inside the actual service container: ```bash podman exec bash -c 'exec 3<>/dev/tcp/host.containers.internal/ && echo ALIAS-OK || echo ALIAS-FAIL' ``` Note: busybox `nc -z` reports any failure as failure — use it for bulk port sweeps; use the `/dev/tcp` probe (or `nc -v`) when the refusal-vs-timeout distinction matters. ## Fix decision - If the alias path connects and the LAN-IP path refuses → set the service's host-facing URL to `http://host.containers.internal:` in its compose/quadlet env, then redeploy via the normal git flow (`deploy.sh --service ` recreates the container with fresh env). - Do NOT publish the daemon port just to make the LAN IP work — publishing is exactly what makes other ports refuse. ## Post-fix verification `podman exec printenv ` then tail logs for the connect confirmation (e.g. `Signal: connected`, `SSE: connected`) within ~40 s of recreation.