Files
command-center/agent/managed-skills/podman-netavark-host-port-probe/SKILL.md
T
Frank 9bfa0be85d chore(skills): promote runbook skills to managed-skills; drop superseded SKILL.md
- 31 new managed skills (anubis/arr/authentik/hermes/freetoken/podman/llm-swap
  runbooks etc.) now tracked.
- Removed superseded skills: omp_harness_config, peon_ping_integration
  (replaced by user-authored peon-ping-* managed skills),
  rocmfp4-strix-benchmark-orchestration.
- New memory-root skills: niri_compositor_ops, omp_git_workflow,
  omp_model_config, quadlet-media-migration, homelab-service-triage,
  hermes-dankclaw-ops, frank-default-large-ops, caddy-auth-dns-playbook.
2026-09-09 16:20:28 -07:00

2.2 KiB

name, description
name description
podman-netavark-host-port-probe Diagnose "connection refused" when a rootless-podman container tries to reach a host service port on this netavark host; proves the published-port-only quirk and picks the working path (host.containers.internal).

Podman/netavark: container → host port connectivity probe

Use when a containerized service on a rootless-podman + netavark host cannot reach a HOST process by LAN IP (connection refused immediately, not a timeout) even though ss -ltnp proves the listener is up in the host netns.

Why this happens

On netavark-backed networks, container→host-LAN-IP traffic is only delivered for ports that have a live port-forward/DNAT (published ports). Un-published host ports refuse instantly — there is no firewall rule and no stale DNAT to find; nft list tables/iptables -t nat will be empty. Do not hunt for REJECT rules; probe instead.

Probe recipe (proves the quirk in one command)

ssh <host> "podman run --rm --network <container-network> --entrypoint sh docker.io/library/busybox:latest -c 'for p in <suspicious-port> <published-port>; do nc -w2 -z -v <host-lan-ip> \$p >/dev/null 2>&1 && echo port-\$p-OK || echo port-\$p-BAD; done'"
  • Published port OK + target port BAD ⇒ published-port-only quirk confirmed.
  • Also probe the alias path from inside the actual service container:
podman exec <svc> bash -c 'exec 3<>/dev/tcp/host.containers.internal/<port> && echo ALIAS-OK || echo ALIAS-FAIL'

Note: busybox nc -z reports any failure as failure — use it for bulk port sweeps; use the /dev/tcp probe (or nc -v) when the refusal-vs-timeout distinction matters.

Fix decision

  • If the alias path connects and the LAN-IP path refuses → set the service's host-facing URL to http://host.containers.internal:<port> in its compose/quadlet env, then redeploy via the normal git flow (deploy.sh <target> --service <svc> recreates the container with fresh env).
  • Do NOT publish the daemon port just to make the LAN IP work — publishing is exactly what makes other ports refuse.

Post-fix verification

podman exec <svc> printenv <URL_VAR> then tail logs for the connect confirmation (e.g. Signal: connected, SSE: connected) within ~40 s of recreation.