docs(fleet): radiance standalone engine probe - qwen4exp refuses fp8 input, quant happens at convert

rad-convert --plan-only with the deployed orcarouter FP8 payload:
'no architecture plugin claims qwen4exp at quantisation fp8; served at
(unquantised)'. Our quantization is reproducible from a bf16/abliterated
source by recipe (libr4d reads fp8 experts), not from our file. Image
pulled on dankserver, no GPU touched.
This commit is contained in:
Frank
2026-10-05 14:11:47 -07:00
parent 2e64f66b82
commit b06b47c7b4
@@ -168,11 +168,46 @@ Not advisable.
| **Affinity** | yes, built here | **V4-Flash only** — "the geometry is compiled in; there is no architecture abstraction" | 110 tok/s for 284B. Best engine for this box by an order of magnitude, wrong model |
| **tcclaviger vLLM fork** | **yes — proven 2026-09-29** | qwen4exp measured 37.4-40.6 tok/s decode (see [[dankserver-qwen38-fp8-expert-offload-tuning]]); **`glm5next` registered** (`Glm5NextForCausalLM`/`MTP`, backend-split `amd/`/`common/`/`nvidia/`, `sparse_indexer.py` for DSA); r4d.so has **native mxfp4 + dsfp4 MoE entries** (`has_dsfp4() == True`); W4A16 MoE class (`R4dW4a16MoEExperts`) wired into the same tiered offload; TunableOp silently skipped under offload (needs `CLAV_TUNABLEOP_WITH_OFFLOAD=1`, and then corrupts numerics — do not use) | the ad hoc qwen-fp8 container's runtime; **the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead** — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels |
| **Lucebox** | yes, image on host | Qwen 3.8, DS4; **V4.1 on the unmerged `feat/ds41-engine` branch** | The only V4.1 path. **Refuses layer split and layer-major prefill for V4.1** |
| **radiance-vLLM** (drwolfen) | yes — the active frank-primary | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture |
| **radiance-vLLM** (drwolfen) | yes — served frank-primary 2026-09-27 → 09-30 | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture |
| **FreeToken-ROCm** | partial | block-FP8 safetensors only (modelopt); known `CpuMoeExecutor` issue under ROCm | cannot load a GGUF |
| **llama.cpp** | yes | GLM-5.3 confirmed by a community R9700 guide; DS-V4.1 CED unverified | no expert LRU; **8 CPU cores** make CPU expert placement the wrong trade |
| **TokenSpeed** | claimed | GLM-5.x incl. a 5.3-Flash recipe | NVIDIA-first, unverified on gfx1201 |
| **KTransformers** | no | GLM-5.3 tutorial exists | built for Intel AMX + NVIDIA |
| **radiance** (StillDeadcode — the standalone C++/HIP engine, NOT the vLLM fork of the same name) | image `stilldeadcode/radiance` prebuilt for gfx1201 | qwen35 dense/MoE at fp8, qwen4exp quantised-at-convert, llama-shaped | see the probe below: our FP8 payload is REFUSED; tiers, MTP and vision native |
### radiance (standalone) — 2026-10-05 probe
`codeberg.org/StillDeadcode/radiance` — Deadcode's from-scratch engine
(v1.0.13): same author as the `vllm-radiance` fork we ran in September
([[frank-medium-radiance-mxfp4]]), an unrelated codebase — C++/HIP, no
Python, kernels/architectures/quantisers as `.so` plugins, flags written
for two 32 GB R9700s, native Qwen3.8-Flash-Next with `--placement
expert_tiered`, MTP and vision.
Probe with the DEPLOYED payload (`rad-convert --plan-only`, which stops
at declare — no GPU touched, no bytes written):
```
E no architecture plugin claims 'qwen4exp' at quantisation 'fp8'.
'qwen4exp' is served at: (unquantised).
```
So the 173 G orcarouter FP8 file cannot enter this engine at all: the
qwen4exp plugin takes bf16/f16/f32/GGUF in and every quantisation happens
at `rad-convert` time by recipe. Not a lock-in to their published quant —
recipes are per-weight-class rules (`rtn`/`gptq`/`awq`/`autoround`/
`paroquant`/`ggml`, grids from fp8 128×128 to i4/w4nl/w5nl codebooks,
custom `--quant` rules go ahead of the file's), and libr4d lists routed
experts-as-fp8 among the encodings it reads — so today's quantization is
reproducible **from a bf16 source**, not from our file. Uncensored-family
bf16 source candidates: `windowsxp811203/Qwen3.8-Flash-Next-Abliterated-GGUF`
(ships the bf16 GGUF) and `mradermacher/Qwen3.8-Flash-Next-Uncensored-GGUF`
(i1 quants only). 622 G free fits source + container. A trial is:
download → `rad-convert` (CPU-side) → stop+mask frank-primary window to
serve. Caveats on record: `--tp-wire wht6` is a lossy int8 all-reduce
wire; the recurrent GDN state is f32 by default (144 MiB/sequence) with
an opt-in f16 that changes generated text; every pool is budgeted
explicitly — no profiling pass, no utilisation fraction.
The two-R9700 three-owner split (primary + secondary + stream) is **not
available on this host**: the retired quadlet recorded that