docs(LLM): recipe portability audit — qwen35moe plugin carries the calibration tap; 35B-A3B w4 fits one R9700 resident
This commit is contained in:
@@ -162,6 +162,35 @@ REAP the expert set BEFORE step 3; ablate BEFORE calibration.
|
||||
Native encodings beyond w4: i8 trunk, fp8 HC/PLE/KV, bf16 protected,
|
||||
u2 draft — all recipe-expressible (see the w4dn8 runbook).
|
||||
|
||||
#### Recipe portability: the w4dn8 shape onto other MoEs (2026-10-08 audit)
|
||||
The pin 1.0.13 container ships SIX arch plugins
|
||||
(`/opt/radiance/share/radiance/architectures/`): `qwen4exp_fp8`,
|
||||
**`qwen35moe_fp8`**, `qwen35_fp8`, `qwen35_bf16`, `llama_fp8`,
|
||||
`llama_dense_fp8` (census via `podman run --rm --entrypoint sh <image> -c`
|
||||
— the image has no `ls`/`strings`; grep the `.so` files directly, and
|
||||
the image lives in ROOTLESS storage as `docker.io/stilldeadcode/radiance`).
|
||||
`grep -c` receipts: `qwen35moe_fp8.so` carries the `RADIANCE_CALIB` tap
|
||||
(2) and gram export (2) plus MTP (4) and NO ngram (0) — i.e. the full
|
||||
calibration→GPTQ path exists for the qwen3.5-MoE family; GPTQ itself
|
||||
lives in `rad-convert`, not the plugin (0 hits in both `.so`).
|
||||
|
||||
Worked example — `Qwen/Qwen3.6-35B-A3B` (config: `qwen3_5_moe_text`,
|
||||
40 layers, `full_attention_interval=4` DeltaNet hybrid, **256 experts
|
||||
top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
|
||||
248320, multimodal wrapper with vision tower):
|
||||
- Expert set ~32 B params → w4 ≈ 16 GiB; trunk (attn/DeltaNet/embed/
|
||||
lm_head ~3 B) at i8 ≈ 3 GiB ⇒ **~20 GiB total — fully VRAM-resident
|
||||
on ONE R9700** (32 GiB card): no TP=2, no host pool, no disk tier,
|
||||
no `--ngram-placement`, no 47.7 GiB PLE copy, no HC term. The recipe
|
||||
loses three lines (PLE/HC/protected-count scales with 256 experts).
|
||||
- Convert cost scales with the file: Flash-Next's 335 GiB took 5 h 50 m
|
||||
host-backend ⇒ ~70 GiB BF16 ≈ 1–1.5 h. KLD reference boot is trivial
|
||||
(BF16 ~65 GiB in host pool beside the pool KV needs).
|
||||
- Unresolved before any real campaign: does `qwen35moe` decode the
|
||||
`Qwen3_5MoeForConditionalGeneration` vision wrapper (assume text-only
|
||||
until proven), and does its MTP cast to the fp8/u2 encodings like
|
||||
qwen4exp. The MNBT-2048 disk-tier law is MOOT for a resident model.
|
||||
|
||||
### Strix Halo — frank-large (halogen) and friends
|
||||
- `.hgn`: **closed production door.** No public quantizer exists
|
||||
(verified absence across peonist-ai repos); the format's lock is the
|
||||
|
||||
Reference in New Issue
Block a user