docs: GLM-5.3-Flash readiness revised — tcclaviger fork has native MXFP4/DSFP4
Source reading in the qwen-fp8 container: glm5next is registered with an amd/ backend dir + DSA indexer; r4d.so has native moe_dsfp4/moe_dsfp4_mt1 entries (has_dsfp4()==True), so the old 'MXFP4 = emulation, missing kernels' verdict is dead for this fork. DSFP4 = MXFP4 weights + DeepSeek activation law. Offload recipes exist for mxfp4 and W4A16 (R4dW4a16MoEExperts in offloader/expert.py:114). GLM gate is now the payload format (compressed-tensors MXFP4/W4A16, none public) + a HIP boot probe.
This commit is contained in:
1 parent
57072a44df
commit
f532e03bf4
2 files changed
+19
-1
No files matched your search
@@ -105,6 +105,23 @@
|
||||
band, straddling the 35 target within noise. Final 256k config:
|
||||
`MAXLEN=262144 KV_BYTES=4328521728`, prefetch off. The full-speed
|
||||
alternative stays `MAXLEN=184224` at 3 GiB (37.4-40.6 band).
|
||||
- **GLM-5.3-Flash readiness REVISED by source reading (Frank's question
|
||||
about int4 vs mxfp4):** the tcclaviger fork in the qwen-fp8 container
|
||||
registers `Glm5NextForCausalLM`/`Glm5NextMTP` with a backend-split
|
||||
module (`glm5next/amd|common|nvidia`, `sparse_indexer.py` for DSA);
|
||||
`offloader/expert.py` accepts `R4dDsfp4MoEExperts, R4dW4a16MoEExperts,
|
||||
Fp8HipMoEExperts` — and **`R4dDsfp4MoEExperts` is MXFP4 weights with
|
||||
the DeepSeek activation law** (docstring; SwiGLU `gemm1_clamp_limit`
|
||||
default 10.0), inheriting the mxfp4 repack/GEMM/emulation-fallback.
|
||||
**`r4d_lib.has_dsfp4()` is True** in this build — native `moe_dsfp4` /
|
||||
`moe_dsfp4_mt1` entries exist, so the old "MXFP4 = emulation, missing
|
||||
kernels" GLM verdict is dead for this fork. WNA16 (int4) backend
|
||||
oracle on ROCm reduces to HUMMING/TRITON (Marlin/TRTLLM/XPU are
|
||||
CUDA/NVIDIA/Intel-only), payload must be compressed-tensors W4A16
|
||||
group-wise symmetric. Remaining GLM gate: a compressed-tensors MXFP4
|
||||
(~165 GiB) or W4A16 payload + a glm5next-on-HIP boot probe; community
|
||||
GLM-5.3 quants are GGUF (llama.cpp path only), native FP8 is 328 GB
|
||||
(> 246 GiB RAM). Feasibility doc rows updated.
|
||||
|
||||
## 2026-09-28
|
||||
- **Dankclaw model cleanup** (disk was at 93%, 137 G free). Frank asked to
|
||||
|
||||
Reference in new issue
Block a user