frank-primary cutover: GLM-5.3-Flash W4A16 int4 (Intel AutoRound)

Payload: Intel/GLM-5.3-Flash-W4A16-AutoRound (AutoRound symmetric int4 group
128, auto_round:auto_gptq, 34 shards + 2 extras, 113,074 tensors, ~170 GiB).
It is the only W4A16 checkpoint this fork can serve with expert offload: its
quant_method "auto-round" is overridden to "inc", which routes the MoE to
libr4d's grouped W4A16 GEMM (R4dW4a16MoEMethod). A compressed-tensors W4A16
build of the same model resolves to TritonWNA16Experts, which has no offload
recipe, and refuses to serve at all (170 GiB of experts cannot fit 64 GiB of
VRAM, so offload is mandatory).

Shape: tcclaviger RDNA4 vLLM fork (same digest-pinned image as the 2026-09-29
qwen cutover), TP=2, 131,072 ctx x 2 seqs, bf16 KV (fp8 is ILLEGAL here — it
disables the sparse MLA backend's rope-free Triton path, the only MLA decode
gfx12 has), util 0.98, NBATCH 2048, --enable-expert-offload
--expert-precise-memory, prefix caching, glm47 tool + reasoning parsers,
MTP and expert prefetch absent by measurement.

Measured: decode 9.9-10.0 tok/s, prefill 918 tok/s (warm), arena 14.51 GiB =
2,383 slots/rank over 12,096 host-backed experts (73.64 GiB pinned/rank).
THE 35 tok/s TARGET IS NOT MET and the shortfall is measured: live telemetry
(R4D_LRU_TELEMETRY=1) gives a 0.53-0.66 per-layer hit rate, a step moves
0.82 GiB of misses in ~100 ms = 8.2 GB/s effective against a link measured at
22.74 GB/s — the same structural gather-path gap the qwen campaign found.
Growing the arena 13.08 -> 14.51 GiB changed decode by nothing.

Rejected by measurement: MTP4 (head loads, acceptance 3.76-4.90, but decode
unchanged and prefill collapses 918 -> 255 — a verification forward gathers the
union of the batch's experts), NBATCH 8192 (identical prefill).

Also: scripts/patches/gpu_worker-kv-cap.patch (fork-local fix for a KV-reserve
model that under-counts GLM's per-sequence KDA state; not needed by this shape),
runbook, AGENTS.md row + section, both indexes, work log.

Rollback: 2339fae (qwen FP8 + MTP4).

serverAppConfigs-docs/log.md

@@ -1,5 +1,47 @@
 # Bundle Change Log
 
+## 2026-09-30
+- **frank-primary cut over to GLM-5.3-Flash W4A16 int4 — it serves, but NOT at
+  the 35 tok/s target, and the shortfall is measured rather than inferred.**
+  - Payload `Intel/GLM-5.3-Flash-W4A16-AutoRound` (AutoRound symmetric int4
+    group 128, `auto_round:auto_gptq` packing, ~170 GiB, 34 shards + 2 extras,
+    113,074 tensors). It is the only W4A16 checkpoint this fork can serve WITH
+    expert offload: its `quant_method: auto-round` is overridden to `inc`, which
+    routes the MoE to libr4d's grouped W4A16 GEMM (`R4dW4a16MoEMethod`). The
+    compressed-tensors W4A16 build of the same model (`dealignai/…-CYBERSECURITY-W4A16`,
+    182 GiB, fetched earlier) resolves to `TritonWNA16Experts`, which has NO
+    offload recipe — and 170 GiB of experts cannot fit 64 GiB of VRAM, so it
+    refused to serve at all. That checkpoint was purged with the other retired
+    payloads.
+  - Shape: tcclaviger RDNA4 vLLM fork (same digest-pinned image as the qwen
+    cutover), TP=2, 131,072 ctx x 2 seqs, bf16 KV, util 0.98, NBATCH 2048,
+    `--enable-expert-offload --expert-precise-memory`, prefix caching, vision +
+    `glm47` tool and reasoning parsers, MTP OFF, prefetch OFF.
+  - Measured on the deployed engine: **decode 9.9-10.0 tok/s, prefill 918 tok/s**
+    (warm, 2.6k prompt), KV 138,561 tokens; arena 14.51 GiB = 2,383 slots per
+    rank over 12,096 host-backed experts (73.64 GiB pinned per rank).
+  - **The wall:** live expert-cache telemetry (`R4D_LRU_TELEMETRY=1` + a
+    `with_stack=false` profiler window) puts the cache hit rate at 0.53-0.66 per
+    layer; a step moves 0.82 GiB of misses in ~100 ms = **8.2 GB/s effective
+    against a link measured at 22.74 GB/s** — the same structural gather-read-path
+    gap the 2026-09-29 qwen campaign found. Growing the arena 13.08 -> 14.51 GiB
+    (+9%) changed decode by *nothing* (10.03 -> 9.88 tok/s), so residency stopped
+    being the binding term; 35 tok/s would need a 0.89 hit rate.
+  - Measured and rejected: **MTP4** (the head loads, arch resolves to
+    `Glm5NextMTPModel`, acceptance is excellent at mean length 3.76-4.90 — but
+    decode was unchanged (9.6-11.2 vs 10.0) while prefill COLLAPSED 918 -> 255,
+    because a verification forward gathers the *union* of the batch's experts);
+    **fp8 KV** (disables the sparse MLA backend's rope-free Triton path, and
+    aiter's MLA decode op is gated on CDNA>2, so the MTP draft step demands an op
+    that cannot exist on gfx12); **NBATCH 8192** (boots, identical prefill).
+  - Fork bug found: the plan's KV reserve under-models this model's
+    per-sequence KDA state and the MTP draft's KV, and `gpu_worker` caps KV at
+    that reserve, making a legal `--max-model-len` unbootable. Shipped
+    `scripts/patches/gpu_worker-kv-cap.patch` (lets an explicit operator KV
+    request raise the cap) — NOT needed by the deployed shape.
+  - Rollback: commit `2339fae` (qwen FP8 + MTP4); the qwen payload and the R9V
+    state dirs are intact on disk.
+
 
 ## 2026-09-29
 - **frank-large tuning investigation: a measured NEGATIVE result, nothing