Frank
a34322aeb0
The managed-skill library was regenerated: 94 narrow per-incident skills
replaced by 25 broader ones (radiance-*, llamacpp-*, homepage-*, fleet/
endpoint ops). Git renders this as mass delete + mass add because whole
directories changed name; old content stays in history at 12c5e88.
Surface check: the 25 on disk are exactly the skills the harness advertises
this session.
27 lines
2.8 KiB
Markdown
27 lines
2.8 KiB
Markdown
---
|
|
name: radiance-quantize-and-score
|
|
description: "Design a rad-convert recipe for a radiance (StillDeadcode) MoE checkpoint, calibrate it, and score it with the KL mode — the procedure plus the servable-format law."
|
|
---
|
|
|
|
# Radiance: quantize, calibrate, score
|
|
|
|
For a HuggingFace checkpoint → `.rad` on the radiance engine (Codeberg StillDeadcode/radiance).
|
|
|
|
## 1. Know the servable formats before writing a rule
|
|
The block/plugin serves a fixed list; the converter refuses an encoding no kernel reads (`docs/TOOLS.md:184`). For qwen4exp MoE experts: `fp8a8, w4a8, w4a8h, bf16, w4nla8h, w4nl64a8h, w4nl32a8h, w5nl64a8h` (`arch/common/rad_block_moe_fp8.h` `kServed`); dense: `fp8a8, i8a8, fp8a16, mxfp4a8, bf16`, plus the `u2` draft head. `fp6_*`/`u6` encode but never serve. Read `kServed` in source — never trust a code-name list from docs prose.
|
|
|
|
## 2. Recipe file
|
|
Header `#` lines are stored in the container; first matching rule wins, so put `cast dtype=bf16` protected-expert rules and projection-specific rules **ahead of** the general `blk.*.ffn_*_exps.*.weight` rule. Unmatched weights are served as the checkpoint holds them (bf16). Verify names against radiance's internal stems (`ffn_gate_up_exps`, `ffn_down_exps`, `attn_qg`, `ssm_inz`, `ple_ngram`, `output.weight`), not the HF names.
|
|
|
|
## 3. Calibrate (GPTQ/awq/autoround/paroquant need `$CALIB`)
|
|
Bootstrap: `rad-convert` with plain `rtn` → serve with `RADIANCE_CALIB_DIR` (+`RADIANCE_CALIB_TOKENS`) and drive prefill → re-convert with `--recipe … --reuse <bootstrap.rad>` (`--reuse` copies every weight whose encoding did not change). A BF16 checkpoint may be calibrated as it ships: convert with no recipe + `--max-ctx 4096`, serve `--placement expert_tiered --weights-disk-tier`, MNBT large (a step costs the same at 1 token as at 32K). Corpus must look like production traffic; score on a held-out slice.
|
|
|
|
## 4. Score
|
|
`radiance --model BF16.rad … --kld-record DIR --kld-corpus C.jsonl` (exact wire, bf16 KV) once, then `--kld-ref DIR --kld-out R.json` per candidate. Report = mean/median/p99/p99.9/p99.99 KL, top-1 agreement, both perplexities, **per `source`** — label the corpus's `source` field to get per-turn-type scopes. KLD corpus disjoint from calibration. Keep `--max-num-batched-tokens ≤ 8192` (the logits buffer is ~2 GiB/rank per 4096 tokens).
|
|
|
|
## 5. Set the gate from the published ladder, not from a wish
|
|
Compare against `docs/MOE-W4.md`'s table for the arch (served tier / heavier tier / bf16 floor). A threshold below the bf16 floor rejects every quantized build by construction. Escalation ladder reuses the same Grams (Hessians are grid-independent) → escalate by re-converting only.
|
|
|
|
## 6. Verify before serving
|
|
`rad-tune` at the served shape (TP, spec depth) with a persistent cache mount; read the boot ledger for residency; run the fleet probes.
|