command-center/agent/managed-skills/radiance-w4-campaign/SKILL.md

name, description

name	description
radiance-w4-campaign	Run a radiance GPTQ-w4 quantization campaign on dankserver: calibration tap pin and engagement proof, protected experts, convert, KLD gate, deploy — plus the tap-disk and orphan-engine laws.

Pipeline for quantizing a BF16 MoE checkpoint into a servable .rad on dankserver (frank-primary). Reference run: serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md; scripts in scripts/ (run-swift15-bf16.sh, run-<model>-bf16.sh, calib-push.py, derive-protected-experts.py, apply-protected-cast.sh, convert-<model>.sh, kld-gate-<model>.sh, fleet-probes-<model>.sh).

Sequence:

1. Fetch checkpoint with per-shard LFS sha256 verification (-f, byte counts, no token on argv).
2. BF16 container: rad-convert no-recipe (pure copy) — serves as the KLD reference engine.
3. KLD reference: record logits on the SERVE pin at the production geometry, --max-model-len under the pool, corpus ≤ that window. Writes atomically at completion (rows: line = receipt).
4. Calibration tap — see the pin law and the engagement test below.
5. Protected experts: derive from gram energy (share ≥3 % AND ≥3× routing share), splice between # PROTECTED-BEGIN/END markers of the recipe. Plugin law: ONE encoding per expert table; only whole bf16 experts legal beside it — no gate_up/down split.
6. Final convert on the SERVE pin (CPU GPTQ, hours; disk gate ≥146 GiB free first).
7. KLD gate on the serve pin at the production MNBT (a w4 does not stream in place outside the shape it was measured at), pool and KV dtype matching production. Gate: mean KL ≤ 0.03, top-1 ≥ 95 %.
8. Deploy ./scripts/deploy.sh dankserver --service frank-archon; verify /v1/models id = served-model-name, boot ledger, streamed ≥128-token decode probe, fleet probes (vision needs ≥512-token budget + reasoning_content fallback).

The tap pin law (measured 2026-10-30, cost: one 17-hour night)

RADIANCE_CALIB_DIR + RADIANCE_CALIB_TOKENS arms the per-expert tap on the 1.2.x line only.

• Every plugin from 1.2.4 through 1.8.1 still contains the machinery (CalibExperts, RADGRAM2, calib_write_host, the seal line calibration wrote layer Hessians after %lld tokens), so strings proves nothing about the pin.
• A runner that boots FRANK_ARCHON_IMAGE (1.8.1) for calib serves perfectly — /health 200, 1,700 documents at 118 tok/s, .rig-boot written, no error anywhere — and writes zero bytes: the tap never arms.
• The engine exposes no tap counter in /stats and prints no arming line in the boot ledger, so a dead tap is indistinguishable from a slow one for hours.
• Default calib to the 1.2.0 digest (stilldeadcode/radiance@sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa on dankserver as of 2026-10-30); override only by naming --image on purpose.

Engagement test — 3 minutes, run it before any long tap: boot with RADIANCE_CALIB_TOKENS=8192, push 2–3 real corpus documents, then require the seal line and bytes in the tap dir. On 1.2.0 that reads calibration wrote layer Hessians after 10207 tokens to /data/calib. No seal line in ten minutes = the pin or the arming is wrong; stop.

The tap's disk and process laws

• One seal writes ~40 GB (39.4 GiB measured for an 8,192-token floor): the output is expert-shaped, not token-shaped. Put the tap dir on a volume with ≥60 GiB free and gate it before boot (Q38_CALIB_MIN_FREE_MIB). On a 37 GiB root fs it filled / to 0 bytes; the engine's later writes came back cannot write / write was short, and other tooling on the host then failed too.
• Removing the container does not kill its engine. podman rm -f <rig> leaves podman-init -- radiance … alive holding /dev/kfd; production then fails ar_oneshot_2rank_wht6 … device error at kernel init, exits 0, and Restart=always loops it (counter reached 100) while /health never answers. Diagnose with sudo fuser -v /dev/kfd; kill by pid from fuser /dev/kfd + /proc/<pid>/comm == radiance, never pkill -f (the pattern matches the caller's own argv). Put that gate in the runner's restore trap.
• The service stop/restore path needs sudo systemctl — polkit denies the unprivileged call intermittently (Access denied … interactive authentication), and a failed stop means the rig boots beside production.

Rate measurement

/stats prefill_tps is a rolling gauge, never a cumulative counter: gauge × uptime invents rates (it once produced a bogus 4.5×). Derive tok/s from total_requests deltas × the corpus mean tokens/document, or from steps_prefill.

Laws: stop production for every GPU phase; verify shapes against podman inspect <name> --format {{join .Config.Cmd " "}} (rootless, as dank) before writing them; ff-pull fails on untracked files the merge tracks — trash first.