command-center/agent/managed-skills/container-port-image-identity-binding/SKILL.md

name, description

name	description
container-port-image-identity-binding	Diagnose or write fleet gates that silently verify the WRONG process/image: /health answered by a dying container, podman inspect naming a merely-created container, cgroup lacking the service name, RepoDigests[0] vs the pin, detached launchers losing cwd. Bind every fact to its object by host pid.

Container / port / image identity binding

For fleet gates, restart checks, "is it really up?" triage, and any bench/window harness that stops a production service and starts a rig on the same port. Use when a check passes but the thing still misbehaves, or when the same class of silent failure keeps recurring.

The failure signature (all of these mean "you checked a proxy")

Check that lied	What it actually measured
curl /health = 200	whichever process owned the socket, often the one being stopped
podman inspect -f '{{.State.Running}}'	a container that was created, not one that bound a port
podman ps / container name	existence, not service
pgrep -f "<pattern>"	your own polling shell, or a sibling arm
log mtime advancing	some request completed, not necessarily yours
df <dir>	a filesystem that merely looks related to the one the bytes go to
podman image inspect -f '{{index .RepoDigests 0}}'	ONE of several digests — a by-digest pull makes an image carry the tag digest first
rootless cgroup path	the container ID, never the service/container name

The bindings that work

• Listener → process: ss -H -ltnp 'sport = :<port>' → pid=; needs privilege for foreign processes (sudo ss when the listener is root's).
• Host pid → container: podman top <ctr> hpid (header line first). podman top <ctr> pid prints container-internal pids (1, 7) and compares against nothing — this is the single most expensive mistake in this class.
• Image pin: podman image inspect -f '{{range .RepoDigests}}{{println .}}{{end}}' <ref> and test membership of the pin, never position. Message the difference: "present but does not carry X".
• Device holder: fuser /dev/kfd → pids → resolve each through podman top <ctr> hpid; anything outside the allowed set is foreign (an orphaned engine wedges the next boot as device error).
• Detached launcher: assert cwd (cd "$(dirname "$0")/.."; verify a known executable exists) and write a PASS/FAIL line per fact. setsid nohup … & turns a lost cwd into silence, not an error.

Procedure

1. Write ONE preflight that prints PASS <fact> … / FAIL <fact> — reason, exit 2 on any FAIL. Facts: cwd, image digest membership, free space on the real target filesystem, stale artifacts, GPU holders, listener, inputs, reference data.
2. Call it as step 0 of the orchestrator, before any GPU/port is touched. Delete the duplicated inline checks so the fact has one owner.
3. Pin each lie with a bats test using PATH stubs (podman, ss, fuser): wrong digest, foreign listener pid, foreign GPU pid, dirty dir, wrong cwd. Fixture files must be NON-EMPTY — a -s fact on a : > file fixture fails for the wrong reason.
4. Keep one driver. Ad-hoc copies beside the orchestrator drop guards, and two drivers racing on one port produce phantom "strays" that look like a hijack.

Traps

• podman top … pid vs hpid — see above.
• Stub collision: a test stub that answers top for both the rig and production must branch on the container name, or the test proves nothing.
• A gate that reports a stale-but-present image must distinguish "absent" from "retagged/older" — the operator's fix differs.
• Verify a kill by re-reading the port; podman rm -f frees the cgroup, a pattern-kill may not.