frank-primary cutover: GLM-5.3-Flash W4A16 int4 (Intel AutoRound)

Payload: Intel/GLM-5.3-Flash-W4A16-AutoRound (AutoRound symmetric int4 group
128, auto_round:auto_gptq, 34 shards + 2 extras, 113,074 tensors, ~170 GiB).
It is the only W4A16 checkpoint this fork can serve with expert offload: its
quant_method "auto-round" is overridden to "inc", which routes the MoE to
libr4d's grouped W4A16 GEMM (R4dW4a16MoEMethod). A compressed-tensors W4A16
build of the same model resolves to TritonWNA16Experts, which has no offload
recipe, and refuses to serve at all (170 GiB of experts cannot fit 64 GiB of
VRAM, so offload is mandatory).

Shape: tcclaviger RDNA4 vLLM fork (same digest-pinned image as the 2026-09-29
qwen cutover), TP=2, 131,072 ctx x 2 seqs, bf16 KV (fp8 is ILLEGAL here — it
disables the sparse MLA backend's rope-free Triton path, the only MLA decode
gfx12 has), util 0.98, NBATCH 2048, --enable-expert-offload
--expert-precise-memory, prefix caching, glm47 tool + reasoning parsers,
MTP and expert prefetch absent by measurement.

Measured: decode 9.9-10.0 tok/s, prefill 918 tok/s (warm), arena 14.51 GiB =
2,383 slots/rank over 12,096 host-backed experts (73.64 GiB pinned/rank).
THE 35 tok/s TARGET IS NOT MET and the shortfall is measured: live telemetry
(R4D_LRU_TELEMETRY=1) gives a 0.53-0.66 per-layer hit rate, a step moves
0.82 GiB of misses in ~100 ms = 8.2 GB/s effective against a link measured at
22.74 GB/s — the same structural gather-path gap the qwen campaign found.
Growing the arena 13.08 -> 14.51 GiB changed decode by nothing.

Rejected by measurement: MTP4 (head loads, acceptance 3.76-4.90, but decode
unchanged and prefill collapses 918 -> 255 — a verification forward gathers the
union of the batch's experts), NBATCH 8192 (identical prefill).

Also: scripts/patches/gpu_worker-kv-cap.patch (fork-local fix for a KV-reserve
model that under-counts GLM's per-sequence KDA state; not needed by this shape),
runbook, AGENTS.md row + section, both indexes, work log.

Rollback: 2339fae (qwen FP8 + MTP4).

serverAppConfigs-docs/LLM/index.md

@@ -1,10 +1,13 @@
 # LLM Fleet — Knowledge Base
 
-Documentation for the LLM inference fleet on the two hosts (2026-09-27):
-**Dankserver runs R9V radiance-vLLM on frank-primary** (the drwolfen fork
-serving the unsloth Qwen3.8-Flash-Next Q8_0 + MTP4 across both Radeon AI
-PRO R9700s on :7770, docker-pinned runtime driven through a docker→podman
-shim, the Qwen-Sharp template, 131072 ctx), and Dankclaw runs two llama.cpp endpoints —
+Documentation for the LLM inference fleet on the two hosts (2026-09-30):
+**Dankserver runs the tcclaviger RDNA4 vLLM fork on frank-primary** (the
+digest-pinned `tcclaviger/vllm` image driving `podman run` directly, serving
+Intel/GLM-5.3-Flash-W4A16-AutoRound — symmetric int4 group 128 — across both
+Radeon AI PRO R9700s on :7770, TP=2 with the expert set offloaded to 73.64 GiB
+of pinned host RAM per rank behind a 14.51 GiB VRAM LRU arena; 131,072 ctx,
+bf16 KV, vision and tool calls on, measured 9.9-10.0 tok/s decode), and
+Dankclaw runs two llama.cpp endpoints —
 stock llama.cpp (unsloth b11030, ROCm gfx1151; frank-large, unsloth
 Qwen3.8-27B UD-Q8_K_XL + the z-lab DFlash2 Q4_K_M drafter on the gfx1151
 iGPU, port 7770) and mainline llama.cpp (frank-small, HauhauCS Gemma4-12B
@@ -22,11 +25,11 @@ graph LR
     subgraph Dankserver[AMD Dankserver - Ubuntu 26.04]
         A[2x Radeon AI PRO R9700
 gfx1201, 32 GiB each]
-        B2[R9V radiance-vLLM fork, docker-pinned
-via docker-podman shim
-frank-primary, port 7770]
-        C[Qwen3.8-Flash-Next Q8_0
-TP=2 tiered experts, MTP4, sharp template, 131072 ctx, vision]
+        B2[tcclaviger RDNA4 vLLM fork, digest-pinned
+podman run, frank-primary, port 7770]
+        C[GLM-5.3-Flash W4A16 int4
+TP=2, expert offload 73.64 GiB/rank pinned,
+14.51 GiB LRU arena, 131072 ctx, vision, tools]
     end
 
     subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
@@ -58,20 +61,26 @@ frank-small, port 7771]
 
 ## Active Deployment
 
-- [[frank-primary-r9v-radiance-qwen38-q8]] — the ACTIVE frank-primary
-  runbook (2026-09-27, root systemd unit, port 7770): the R9V
-  radiance-vLLM fork (drwolfen/radiance-vllm-qwen4exp, docker-pinned
-  image bundle `2dac17a2`, driven through a docker→podman shim) serving
-  the unsloth Qwen3.8-Flash-Next-GGUF Q8_0 (175.30 GiB, 6 shards) +
-  blockfp8 MTP4 (depth 4) + vision on BOTH R9700s — TP=2 tiered experts
-  (80-slot LRU cache), PLE n-gram PINNED in host RAM (54.4 GB), CED off,
-  131072 ctx, the Qwen-Sharp v22.4.0 template via the launcher's
-  `R9V_CHAT_TEMPLATE_FILE` passthrough (fork `30d4a32`), the quantized
-  inject-shard dev-overlay adapter (fork `363d606`), reasoning channel
-  `reasoning`, parser `qwen3_coder`. The tuned shape lives in the
-  on-host q8xl setup state. Rollback:
-  [[frank-primary-r9v-radiance-qwen38-uncensored]]
-- [[frank-primary-r9v-radiance-qwen38-uncensored]] — RETIRED 2026-09-27
+- [[frank-primary-glm53-flash-w4a16]] — the ACTIVE frank-primary runbook
+  (2026-09-30, root systemd unit, port 7770): the tcclaviger RDNA4 vLLM fork
+  (digest-pinned `tcclaviger/vllm@sha256:da4029666…`, plain `podman run`,
+  no shim) serving Intel/GLM-5.3-Flash-W4A16-AutoRound — AutoRound symmetric
+  int4 group 128, ~170 GiB, 34 shards + 2 extras, 113,074 tensors — on BOTH
+  R9700s, TP=2, with the MoE experts offloaded to pinned host RAM (12,096
+  experts/rank, 73.64 GiB) behind a 14.51 GiB / 2,383-slot VRAM LRU arena; the
+  MoE kernel is libr4d's grouped W4A16 GEMM (`R4dW4a16MoEMethod`, reached
+  because the checkpoint's `quant_method: auto-round` is overridden to `inc` —
+  a compressed-tensors W4A16 checkpoint of the same model has NO offload recipe
+  and refuses to serve). 131,072 ctx x 2 seqs, bf16 KV, util 0.98, prefix
+  caching, vision and `glm47` tool/reasoning parsers, MTP OFF and prefetch OFF
+  by measurement. **Measured decode 9.9-10.0 tok/s, prefill 918 tok/s — the
+  35 tok/s target is NOT met, bounded by the fork's expert-gather read path
+  (~8.2 GB/s effective vs a 22.74 GB/s link) at a 0.53-0.66 cache hit rate.**
+  Rollback: commit `2339fae` (the qwen FP8 + MTP4 shape of 2026-09-29 → 09-30)
+- [[frank-primary-r9v-radiance-qwen38-q8]] — RETIRED 2026-09-30 (two swaps
+  back; payload and R9V state dirs intact on disk, git tag
+  frank-primary-r9v-q8-20260929 is the last pre-cutover commit):
+ [[frank-primary-r9v-radiance-qwen38-uncensored]] — RETIRED 2026-09-27
   (same-day swap to the Q8_0 payload; rollback target, state + payload
   intact on disk): the R9V radiance-vLLM fork serving
   Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS (92.39 GiB, 3 shards) +