diff --git a/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md b/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md index b5df329..12fe489 100644 --- a/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md +++ b/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md @@ -168,11 +168,46 @@ Not advisable. | **Affinity** | yes, built here | **V4-Flash only** — "the geometry is compiled in; there is no architecture abstraction" | 110 tok/s for 284B. Best engine for this box by an order of magnitude, wrong model | | **tcclaviger vLLM fork** | **yes — proven 2026-09-29** | qwen4exp measured 37.4-40.6 tok/s decode (see [[dankserver-qwen38-fp8-expert-offload-tuning]]); **`glm5next` registered** (`Glm5NextForCausalLM`/`MTP`, backend-split `amd/`/`common/`/`nvidia/`, `sparse_indexer.py` for DSA); r4d.so has **native mxfp4 + dsfp4 MoE entries** (`has_dsfp4() == True`); W4A16 MoE class (`R4dW4a16MoEExperts`) wired into the same tiered offload; TunableOp silently skipped under offload (needs `CLAV_TUNABLEOP_WITH_OFFLOAD=1`, and then corrupts numerics — do not use) | the ad hoc qwen-fp8 container's runtime; **the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead** — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels | | **Lucebox** | yes, image on host | Qwen 3.8, DS4; **V4.1 on the unmerged `feat/ds41-engine` branch** | The only V4.1 path. **Refuses layer split and layer-major prefill for V4.1** | -| **radiance-vLLM** (drwolfen) | yes — the active frank-primary | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture | +| **radiance-vLLM** (drwolfen) | yes — served frank-primary 2026-09-27 → 09-30 | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture | | **FreeToken-ROCm** | partial | block-FP8 safetensors only (modelopt); known `CpuMoeExecutor` issue under ROCm | cannot load a GGUF | | **llama.cpp** | yes | GLM-5.3 confirmed by a community R9700 guide; DS-V4.1 CED unverified | no expert LRU; **8 CPU cores** make CPU expert placement the wrong trade | | **TokenSpeed** | claimed | GLM-5.x incl. a 5.3-Flash recipe | NVIDIA-first, unverified on gfx1201 | | **KTransformers** | no | GLM-5.3 tutorial exists | built for Intel AMX + NVIDIA | +| **radiance** (StillDeadcode — the standalone C++/HIP engine, NOT the vLLM fork of the same name) | image `stilldeadcode/radiance` prebuilt for gfx1201 | qwen35 dense/MoE at fp8, qwen4exp quantised-at-convert, llama-shaped | see the probe below: our FP8 payload is REFUSED; tiers, MTP and vision native | + +### radiance (standalone) — 2026-10-05 probe + +`codeberg.org/StillDeadcode/radiance` — Deadcode's from-scratch engine +(v1.0.13): same author as the `vllm-radiance` fork we ran in September +([[frank-medium-radiance-mxfp4]]), an unrelated codebase — C++/HIP, no +Python, kernels/architectures/quantisers as `.so` plugins, flags written +for two 32 GB R9700s, native Qwen3.8-Flash-Next with `--placement +expert_tiered`, MTP and vision. + +Probe with the DEPLOYED payload (`rad-convert --plan-only`, which stops +at declare — no GPU touched, no bytes written): + +``` +E no architecture plugin claims 'qwen4exp' at quantisation 'fp8'. + 'qwen4exp' is served at: (unquantised). +``` + +So the 173 G orcarouter FP8 file cannot enter this engine at all: the +qwen4exp plugin takes bf16/f16/f32/GGUF in and every quantisation happens +at `rad-convert` time by recipe. Not a lock-in to their published quant — +recipes are per-weight-class rules (`rtn`/`gptq`/`awq`/`autoround`/ +`paroquant`/`ggml`, grids from fp8 128×128 to i4/w4nl/w5nl codebooks, +custom `--quant` rules go ahead of the file's), and libr4d lists routed +experts-as-fp8 among the encodings it reads — so today's quantization is +reproducible **from a bf16 source**, not from our file. Uncensored-family +bf16 source candidates: `windowsxp811203/Qwen3.8-Flash-Next-Abliterated-GGUF` +(ships the bf16 GGUF) and `mradermacher/Qwen3.8-Flash-Next-Uncensored-GGUF` +(i1 quants only). 622 G free fits source + container. A trial is: +download → `rad-convert` (CPU-side) → stop+mask frank-primary window to +serve. Caveats on record: `--tp-wire wht6` is a lossy int8 all-reduce +wire; the recurrent GDN state is f32 by default (144 MiB/sequence) with +an opt-in f16 that changes generated text; every pool is budgeted +explicitly — no profiling pass, no utilisation fraction. The two-R9700 three-owner split (primary + secondary + stream) is **not available on this host**: the retired quadlet recorded that