command-center/agent/managed-skills/fleet-service-runtime-census/SKILL.md

name, description

name	description
fleet-service-runtime-census	Answer "what is service running right now?" for a dank-fleet service: unit state, container image/digest, endpoint contract, loaded env — machine over repo, in that order.

Answer "what is service running right now?" for a dank-fleet service without trusting the repo. Repo state is intent; the machine is truth, and they differ by however long since the last deploy.

Order of operations (cheap → expensive)

1. Unit state on the host (rootless quadlet ⇒ --user; some units are root): ssh -o BatchMode=yes host 'systemctl --user show S -p ActiveState -p SubState -p ExecMainStartTimestamp -p NRestarts' Boot-loop tell: NRestarts climbing with a short ExecMainStartTimestamp.
2. The container's own view: podman ps --filter name=S --format '{{.Image}} {{.Status}} {{.Names}}', then podman inspect for Image (tag and RepoDigests) and Env.
3. The endpoint contract, from the service itself — this is the only answer that matters to a consumer: curl -s host:port/v1/models (served model id), curl -s host:port/props (n_ctx, n_ctx_train, caps), curl -s -o /dev/null -w '%{http_code}' host:port/health.
4. Effective flags: the live process is the truth. systemctl --user cat S shows EnvironmentFile=; read that file. A repo scripts/S.env can be ahead of, or behind, what the running unit loaded — and a unit restart is what re-reads it.
5. Logs last: journalctl --user -u S -n 80 --no-pager for the boot ledger / refusal messages, after you know whether it is even up.

Facts that bite

• A service is up ≠ configured how you think. llama.cpp's /v1/models reports max_model_len, not the engine's per-slot ceiling; halogen reports n_ctx per slot. Read /props before quoting any window number.
• systemctl is-enabled answers enablement, not activity. A timerless service reports linked/static; deploy.sh that prints ENABLE for it is lying. Read systemctl is-enabled --quiet exit status, not the text.
• NRestarts: 0 with ActiveState=activating is a crash-loop only if the timestamps move; check ExecMainStartTimestamp twice, 5 s apart.
• Read-only is always allowed from the dev box (ssh -o BatchMode=yes); state changes on remote boxes are not, without a per-run OK. Production services (frank-archon, frank-dragoon, frank-zealot) are never stopped/restarted to "see what happens".
• Digest-pinned images: "when was this image updated?" is answered by podman inspect RepoDigests + the registry API, not by the unit's start time (a restart re-uses the local digest).

Report

One line per layer with the evidence: unit (active (running), since, restarts), container (image + digest), endpoint (served id, n_ctx, health code), loaded env (file path + the knobs that matter), and explicitly which of those came from the machine versus the repo — that gap is the answer to most "why is it behaving like that" questions.