40 lines
4.8 KiB
Markdown
40 lines
4.8 KiB
Markdown
---
|
||
name: radiance-w4-campaign
|
||
description: "Run a radiance GPTQ-w4 quantization campaign on dankserver: calibration tap pin and engagement proof, protected experts, convert, KLD gate, deploy — plus the tap-disk and orphan-engine laws."
|
||
---
|
||
|
||
Pipeline for quantizing a BF16 MoE checkpoint into a servable `.rad` on dankserver (frank-primary). Reference run: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`; scripts in `scripts/` (`run-swift15-bf16.sh`, `run-<model>-bf16.sh`, `calib-push.py`, `derive-protected-experts.py`, `apply-protected-cast.sh`, `convert-<model>.sh`, `kld-gate-<model>.sh`, `fleet-probes-<model>.sh`).
|
||
|
||
Sequence:
|
||
1. **Fetch** checkpoint with per-shard LFS sha256 verification (`-f`, byte counts, no token on argv).
|
||
2. **BF16 container**: `rad-convert` no-recipe (pure copy) — serves as the KLD reference engine.
|
||
3. **KLD reference**: record logits on the SERVE pin at the production geometry, `--max-model-len` under the pool, corpus ≤ that window. Writes atomically at completion (`rows:` line = receipt).
|
||
4. **Calibration tap** — see the pin law and the engagement test below.
|
||
5. **Protected experts**: derive from gram energy (share ≥3 % AND ≥3× routing share), splice between `# PROTECTED-BEGIN/END` markers of the recipe. Plugin law: ONE encoding per expert table; only whole bf16 experts legal beside it — no gate_up/down split.
|
||
6. **Final convert** on the SERVE pin (CPU GPTQ, hours; disk gate ≥146 GiB free first).
|
||
7. **KLD gate** on the serve pin at the production MNBT (a w4 does not stream in place outside the shape it was measured at), pool and KV dtype matching production. Gate: mean KL ≤ 0.03, top-1 ≥ 95 %.
|
||
8. **Deploy** `./scripts/deploy.sh dankserver --service frank-archon`; verify `/v1/models` id = served-model-name, boot ledger, streamed ≥128-token decode probe, fleet probes (vision needs ≥512-token budget + `reasoning_content` fallback).
|
||
|
||
## The tap pin law (measured 2026-10-30, cost: one 17-hour night)
|
||
|
||
`RADIANCE_CALIB_DIR` + `RADIANCE_CALIB_TOKENS` arms the per-expert tap on the **1.2.x** line only.
|
||
|
||
- Every plugin from 1.2.4 through 1.8.1 still **contains** the machinery (`CalibExperts`, `RADGRAM2`, `calib_write_host`, the seal line `calibration wrote layer Hessians after %lld tokens`), so `strings` proves nothing about the pin.
|
||
- A runner that boots `FRANK_ARCHON_IMAGE` (1.8.1) for `calib` serves perfectly — `/health` 200, 1,700 documents at 118 tok/s, `.rig-boot` written, no error anywhere — and writes **zero bytes**: the tap never arms.
|
||
- The engine exposes **no tap counter** in `/stats` and prints **no arming line** in the boot ledger, so a dead tap is indistinguishable from a slow one for hours.
|
||
- Default `calib` to the 1.2.0 digest (`stilldeadcode/radiance@sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa` on dankserver as of 2026-10-30); override only by naming `--image` on purpose.
|
||
|
||
**Engagement test — 3 minutes, run it before any long tap:** boot with `RADIANCE_CALIB_TOKENS=8192`, push 2–3 real corpus documents, then require the seal line *and* bytes in the tap dir. On 1.2.0 that reads `calibration wrote layer Hessians after 10207 tokens to /data/calib`. No seal line in ten minutes = the pin or the arming is wrong; stop.
|
||
|
||
## The tap's disk and process laws
|
||
|
||
- **One seal writes ~40 GB** (39.4 GiB measured for an 8,192-token floor): the output is expert-shaped, not token-shaped. Put the tap dir on a volume with ≥60 GiB free and gate it before boot (`Q38_CALIB_MIN_FREE_MIB`). On a 37 GiB root fs it filled `/` to **0 bytes**; the engine's later writes came back `cannot write` / `write was short`, and *other* tooling on the host then failed too.
|
||
- **Removing the container does not kill its engine.** `podman rm -f <rig>` leaves `podman-init -- radiance …` alive holding `/dev/kfd`; production then fails `ar_oneshot_2rank_wht6 … device error` at kernel init, exits **0**, and `Restart=always` loops it (counter reached 100) while `/health` never answers. Diagnose with `sudo fuser -v /dev/kfd`; kill **by pid** from `fuser /dev/kfd` + `/proc/<pid>/comm == radiance`, never `pkill -f` (the pattern matches the caller's own argv). Put that gate in the runner's restore trap.
|
||
- The service stop/restore path needs `sudo systemctl` — polkit denies the unprivileged call intermittently (`Access denied … interactive authentication`), and a failed stop means the rig boots *beside* production.
|
||
|
||
## Rate measurement
|
||
|
||
`/stats` `prefill_tps` is a **rolling gauge**, never a cumulative counter: `gauge × uptime` invents rates (it once produced a bogus 4.5×). Derive tok/s from `total_requests` deltas × the corpus mean tokens/document, or from `steps_prefill`.
|
||
|
||
Laws: stop production for every GPU phase; verify shapes against `podman inspect <name> --format {{join .Config.Cmd " "}}` (rootless, as dank) before writing them; ff-pull fails on untracked files the merge tracks — trash first.
|