docs: GLM-5.3-Flash readiness revised — tcclaviger fork has native MXFP4/DSFP4

Source reading in the qwen-fp8 container: glm5next is registered with an
amd/ backend dir + DSA indexer; r4d.so has native moe_dsfp4/moe_dsfp4_mt1
entries (has_dsfp4()==True), so the old 'MXFP4 = emulation, missing
kernels' verdict is dead for this fork. DSFP4 = MXFP4 weights + DeepSeek
activation law. Offload recipes exist for mxfp4 and W4A16 (R4dW4a16MoEExperts
in offloader/expert.py:114). GLM gate is now the payload format
(compressed-tensors MXFP4/W4A16, none public) + a HIP boot probe.
This commit is contained in:
Frank committed 2026-09-29 23:28:04 -07:00
1 parent 57072a44df
commit f532e03bf4
2 files changed
+19 -1

No files matched your search

+17
View File
@@ -105,6 +105,23 @@
band, straddling the 35 target within noise. Final 256k config:
`MAXLEN=262144 KV_BYTES=4328521728`, prefetch off. The full-speed
alternative stays `MAXLEN=184224` at 3 GiB (37.4-40.6 band).
- **GLM-5.3-Flash readiness REVISED by source reading (Frank's question
about int4 vs mxfp4):** the tcclaviger fork in the qwen-fp8 container
registers `Glm5NextForCausalLM`/`Glm5NextMTP` with a backend-split
module (`glm5next/amd|common|nvidia`, `sparse_indexer.py` for DSA);
`offloader/expert.py` accepts `R4dDsfp4MoEExperts, R4dW4a16MoEExperts,
Fp8HipMoEExperts` — and **`R4dDsfp4MoEExperts` is MXFP4 weights with
the DeepSeek activation law** (docstring; SwiGLU `gemm1_clamp_limit`
default 10.0), inheriting the mxfp4 repack/GEMM/emulation-fallback.
**`r4d_lib.has_dsfp4()` is True** in this build — native `moe_dsfp4` /
`moe_dsfp4_mt1` entries exist, so the old "MXFP4 = emulation, missing
kernels" GLM verdict is dead for this fork. WNA16 (int4) backend
oracle on ROCm reduces to HUMMING/TRITON (Marlin/TRTLLM/XPU are
CUDA/NVIDIA/Intel-only), payload must be compressed-tensors W4A16
group-wise symmetric. Remaining GLM gate: a compressed-tensors MXFP4
(~165 GiB) or W4A16 payload + a glm5next-on-HIP boot probe; community
GLM-5.3 quants are GGUF (llama.cpp path only), native FP8 is 328 GB
(> 246 GiB RAM). Feasibility doc rows updated.
## 2026-09-28
- **Dankclaw model cleanup** (disk was at 93%, 137 G free). Frank asked to