Frank
a34322aeb0
The managed-skill library was regenerated: 94 narrow per-incident skills
replaced by 25 broader ones (radiance-*, llamacpp-*, homepage-*, fleet/
endpoint ops). Git renders this as mass delete + mass add because whole
directories changed name; old content stays in history at 12c5e88.
Surface check: the 25 on disk are exactly the skills the harness advertises
this session.
30 lines
2.6 KiB
Markdown
30 lines
2.6 KiB
Markdown
---
|
||
name: radiance-w4-convert-playbook
|
||
description: "Convert a BF16 Qwen3.8-Flash-Next checkpoint to a GPTQ-w4 radiance .rad: recipe law, calibration tap, derive-protected-experts, convert gates, KLD gate shape and flags."
|
||
---
|
||
|
||
# Radiance w4 convert playbook (dankserver)
|
||
|
||
End-to-end recipe for rad-convert GPTQ w4 on the qwen4exp arch (engine pin `stilldeadcode/radiance@sha256:4c3f48...` = 1.0.13 for convert/KLD/serve; 1.2.0 `sha256:665775...` ONLY for the calibration tap).
|
||
|
||
## Pipeline
|
||
1. **Calibration corpus** ~10M tokens (`make-calib-corpus.py`, seed-pinned; keep `kld.jsonl` for the gate — regenerate deterministically if lost).
|
||
2. **KLD reference on the BF16 container** (no-recipe convert, `--kld-record`), same engine as scoring.
|
||
3. **Calib serve on 1.2.0**: `RADIANCE_CALIB_DIR` + `RADIANCE_CALIB_TOKENS=10000000`; tap in `qwen4exp_fp8.so` writes `gram.r<rank>.blk.L.ffn_down_exps.<e>.bin` (RADGRAM2, 32 B header + k×k f32); DONE mark `calibration wrote`.
|
||
4. **Derive protected experts** from the Grams: energy share ≥3 % AND ≥3× routing share, even count per rank half.
|
||
5. **Recipe law**: ONE encoding per expert table — all routed experts through GPTQ w4 (`libquant: GPTQ quantises 'blk.N.ffn_gate_up_exps' against M calibration file(s)`); PROTECTED section = whole bf16 experts ONLY, spliced between `# PROTECTED-BEGIN/END` markers. The $CALIB env var is expanded by rad-convert.
|
||
6. **Convert** host-backend (no GPU; prod-compatible), ~6 min/layer × 49, CPU-saturating. In-container paths must include subdirs (mount prefix strip). Gate: rad-info shows gptq rule; size floor from geometry, not by analogy.
|
||
|
||
## KLD gate shape (1.0.13)
|
||
`--placement expert_tiered --weights-disk-tier --host-pool-mib 40960 --deterministic --gpu-headroom-mib 512 --num-speculative-tokens 0 --kv-cache-dtype fp8 --max-model-len 16384 --ngram-placement ram --kld-ref DIR --kld-out FILE --vram-kv-mib 2048 --max-num-seqs 4 --max-num-batched-tokens 3072`.
|
||
- `--kld-ref` takes NO `--kld-corpus` (rejects the pair).
|
||
- w4 arena ≈ 262.5 MiB per 256 tokens of MNBT — 32768 needs 33 GiB of VRAM.
|
||
- w4 gate_up experts can't disk-tier stream (stored ≠ served form) — pool must hold ~12 GiB/rank.
|
||
- Gate: mean KL ≤ 0.03 AND top-1 ≥ 95 % vs reference.
|
||
|
||
## Pitfalls
|
||
- Editing a running bash script corrupts its execution (lazy byte-offset reads) — sync after the run.
|
||
- `pkill -f <pattern>` kills your own ssh watcher if the pattern matches the ssh command line.
|
||
- zsh on dankserver eats nested quoting — write scripts locally, scp, `sh /tmp/x.sh`.
|
||
- Radiance image has no bash/strings: `--entrypoint sh`, `grep -ac` with positive control.
|