command-center/agent/managed-skills/radiance-calib-window-serve-gate/SKILL.md

name, description

name	description
radiance-calib-window-serve-gate	Diagnose a radiance calibration window that dies at second zero or refuses its push: the health answer coming from the dying production socket, the listener-identity gate, the preflight trio (space, re-chunked corpus, KEEP_RAD), the 12M floor and the wide-step knob.

radiance calibration window — the serve gate and its preflight

Friction-earned 2026-10-30 on dank/serverAppConfigs (scripts/qwen38-calib-window.sh, scripts/run-qwen38-bf16.sh). A 12 M-token window died at second zero; the cause was the gate, not the engine.

The signature, and what it really means

phase=calib health ok waited=0s
the rig container qwen38-bf16 is not running        <- pusher refuses
phase=calib push end rc=2  ... FAIL: serve rc=1 push rc=2

waited=0s is the tell. The runner stops frank-archon first; the dying production process keeps its listener for a moment, while the freshly created rig container is already State.Running=true but has not bound the port yet. So podman inspect (container running) and /health 200 (production's socket) both hold for the wrong process.

The gate must prove the listener belongs to the rig:

lpid=$(ss -H -ltnp 'sport = :7770' | grep -o 'pid=[0-9]*' | head -1 | cut -d= -f2)
[ -n "$lpid" ] || return 1
podman top "$RIG" pid | grep -qw "$lpid"

Landed as PR #90 with two red-first tests (foreign listener ⇒ no push, window fails; in-container listener ⇒ proceeds and banks). If you see waited=0s on a checkout older than that PR, the window there is pre-fix: do not run a long window on it.

Preflight trio — the window refuses correctly, so do not "fix" it

1. Tap dir ≥ 60 GiB free. One seal writes 43 GB in 24,768 files (gram.r<rank>.blk.<l>.ffn_down_exps.<e>.bin). Point it at a 35 GiB root and the seal fills the filesystem to zero. Q38_CALIB_MIN_FREE_MIB overrides — for tests only (fixtures on /tmp tmpfs).
2. The re-chunked corpus. Q38_WINDOW_CALIB_CORPUS defaults to calib-7168.jsonl; pushing calib.jsonl instead runs un-chunked documents through a window sized for chunks.
3. Q38_WINDOW_KEEP_RAD=1 when two windows run back to back. The window banks the BF16 reference container into ~/.AGENTTRASH/ on success; without this, stage 2 has no reference to tap.

Floor and the "add more tokens later" trap

RADIANCE_CALIB_TOKENS is the campaign's number, not the vendor's: the reference container this arm competes with was calibrated on 10 M tokens, so the fleet floor is 12 M. Engine default is 65,536.

A sealed tap cannot be extended. Hessians are VRAM accumulators; the seal writes and drops them. So "run 1 M to verify, then add 11 M" is two windows — 1 M then a fresh 12 M — not an append. Budget accordingly, and rotate the verification tap to .AGENTTRASH/.

One round of the re-chunked corpus is 9.31 M tokens, so a 12 M floor seals in round 1 plus part of round 2; a flat round is normal (the floor is pooled; round_verdict says "continue").

The wide-step knob (the tap's real cost)

Q38_CALIB_MNBT / Q38_CALIB_SEQS set the calibration step and slots; everything else inherits production. The tap is read-bound: a 7,168-token document cut into four 2,048-token steps re-streams each layer's routed expert union four times, so a wider step is worth up to ~4× (the runner's own header says this).

Wide steps are refused by the arch, not the budget, and the refusal scales with the window:

QSA score plane = rows x (--max-model-len / 4) x 4 bytes
16,384 rows @ 200,000 window -> 3,125 MiB  -> REFUSED at boot
16,384 rows @  32,768 window ->   512 MiB  -> legal

The calibration corpus is chunked to 7,168 tokens, so 32,768 is the window to calibrate at — copying production's 200,000 window is what disqualifies the wide step.

Judging a run

• A quiet log is not a hang. The engine logs one line per completed request; at 2-way a 7.5 K prefill takes ~4 min. Watch kv_tokens_used (steps of the chunk size) and the req cmpl count. A watchdog on log mtime kills healthy cycles and throws away accumulated tokens.
• Sealing is success. The engine exits by itself; a mid-flight client gets 429 rate_limit_error: the server is shutting down.
• Kill by pid (sudo fuser /dev/kfd), never pkill -f: an orphaned engine keeps the device and the next boot dies in ~20 s with E startup stopped: device error, which reads like a bad shape.