frank-primary cutover: GLM-5.3-Flash W4A16 int4 (Intel AutoRound)

Payload: Intel/GLM-5.3-Flash-W4A16-AutoRound (AutoRound symmetric int4 group
128, auto_round:auto_gptq, 34 shards + 2 extras, 113,074 tensors, ~170 GiB).
It is the only W4A16 checkpoint this fork can serve with expert offload: its
quant_method "auto-round" is overridden to "inc", which routes the MoE to
libr4d's grouped W4A16 GEMM (R4dW4a16MoEMethod). A compressed-tensors W4A16
build of the same model resolves to TritonWNA16Experts, which has no offload
recipe, and refuses to serve at all (170 GiB of experts cannot fit 64 GiB of
VRAM, so offload is mandatory).

Shape: tcclaviger RDNA4 vLLM fork (same digest-pinned image as the 2026-09-29
qwen cutover), TP=2, 131,072 ctx x 2 seqs, bf16 KV (fp8 is ILLEGAL here — it
disables the sparse MLA backend's rope-free Triton path, the only MLA decode
gfx12 has), util 0.98, NBATCH 2048, --enable-expert-offload
--expert-precise-memory, prefix caching, glm47 tool + reasoning parsers,
MTP and expert prefetch absent by measurement.

Measured: decode 9.9-10.0 tok/s, prefill 918 tok/s (warm), arena 14.51 GiB =
2,383 slots/rank over 12,096 host-backed experts (73.64 GiB pinned/rank).
THE 35 tok/s TARGET IS NOT MET and the shortfall is measured: live telemetry
(R4D_LRU_TELEMETRY=1) gives a 0.53-0.66 per-layer hit rate, a step moves
0.82 GiB of misses in ~100 ms = 8.2 GB/s effective against a link measured at
22.74 GB/s — the same structural gather-path gap the qwen campaign found.
Growing the arena 13.08 -> 14.51 GiB changed decode by nothing.

Rejected by measurement: MTP4 (head loads, acceptance 3.76-4.90, but decode
unchanged and prefill collapses 918 -> 255 — a verification forward gathers the
union of the batch's experts), NBATCH 8192 (identical prefill).

Also: scripts/patches/gpu_worker-kv-cap.patch (fork-local fix for a KV-reserve
model that under-counts GLM's per-sequence KDA state; not needed by this shape),
runbook, AGENTS.md row + section, both indexes, work log.

Rollback: 2339fae (qwen FP8 + MTP4).

tests/frank_primary_unit.bats

@@ -1,19 +1,22 @@
 #!/usr/bin/env bats
 # Seam: frank-primary — the Dankserver LLM endpoint on :7770.
 #
-# This file asserts the tcclaviger RDNA4 vLLM ROOT-UNIT shape (the 2026-09-29
-# cutover that replaced the R9V radiance fork): the serve script drives
-# `podman run` DIRECTLY (the image entrypoint execs `vllm serve`), with every
-# engine flag qualified by boot probe on 2026-09-29 and carried in the repo
-# env file. The seam pins what the flags alone do not say: the qualification
-# law (600k ctx / 3 sessions / q8-q8 KV were MEASURED to fit — 729,545 tokens
-# of capacity at 7.2 GiB per rank — and KV_BYTES is PER RANK), the self-healing
+# This file asserts the tcclaviger RDNA4 vLLM ROOT-UNIT shape (the 2026-09-30
+# cutover to GLM-5.3-Flash W4A16 int4, which replaced the qwen FP8 + MTP4
+# shape): the serve script drives `podman run` DIRECTLY (the image entrypoint
+# execs `vllm serve`), with every engine flag qualified by boot probe on
+# 2026-09-30 and carried in the repo env file. The seam pins what the flags
+# alone do not say: the qualification law (131,072 x 2 seqs at bf16 KV, util
+# 0.98 is what boots; the window is forced up by the fork's KV reserve model,
+# not by capacity), the offload law (170 GiB of experts cannot fit 64 GiB of
+# VRAM — the engine MUST run with --enable-expert-offload), the self-healing
 # law (the run returns before the engine is up, so the serve script must wait
 # on the container for Restart=on-failure to mean anything), the memlock law
-# that makes the tiered-expert pin legal, the deployer's image/payload gates
-# with the false-green MainPID check, and the prefetch law (expert prefetch
-# measured at 36% precision — OFF is the incumbent; a regression that
-# reintroduces it silently costs ~40-50% decode).
+# that makes the 73.64 GiB/rank expert pin legal, the deployer's image/payload
+# gates with the false-green MainPID check, and the three measured-absent
+# flags: expert prefetch (36% precision, costs ~40-50% decode), MTP spec
+# decode (buys zero decode and collapses prefill on an offload-bound shape),
+# and fp8 KV (disables the only MLA decode path gfx12 has).
 
 load helpers
 
@@ -53,8 +56,8 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
     # outlives the unit and holds both GPUs. The container name must match
     # the env file's identity.
     grep -q '^ExecStop=/usr/bin/podman stop' "$UNIT"
-    grep -q 'frank-primary-qwen' "$UNIT"
-    grep -q '^FRANK_PRIMARY_CONTAINER=frank-primary-qwen$' "$ENV_FILE"
+    grep -q 'frank-primary-glm' "$UNIT"
+    grep -q '^FRANK_PRIMARY_CONTAINER=frank-primary-glm$' "$ENV_FILE"
 }
 
 @test "the serve shape is direct podman run, not a launcher or shim" {
@@ -69,32 +72,70 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
 }
 
 @test "the qualified engine shape is in the env file" {
-    # 2026-09-29 boot probe: MAXLEN 200000 x SEQS 3 (600k total) at KV q8/q8
-    # fits — the engine measured "GPU KV cache size: 729,545 tokens,
-    # Maximum concurrency for 200,000 tokens per request: 3.65x" at
-    # KV_BYTES 7.2 GiB per rank. KV_BYTES is PER RANK: sizing it to the
-    # global requirement OOMs the worker (measured twice). fp8 KV is the
-    # requested q8/q8 (e4m3 for both K and V together). MTP4 spec decode,
-    # util 0.98 (0.99 fails the free-memory wall).
-    grep -q '^FRANK_PRIMARY_MAXLEN=200000$' "$ENV_FILE"
-    grep -q '^FRANK_PRIMARY_SEQS=3$' "$ENV_FILE"
-    grep -q '^FRANK_PRIMARY_KV_BYTES=7730934784$' "$ENV_FILE"
-    grep -q '^FRANK_PRIMARY_KV_DTYPE=fp8$' "$ENV_FILE"
-    grep -q '^FRANK_PRIMARY_SPEC=4$' "$ENV_FILE"
+    # 2026-09-30 boot probe: 131,072 x 2 seqs at bf16 KV, util 0.98 boots and
+    # serves (measured "GPU KV cache size: 138,561 tokens"). 0.99 fails the
+    # free-memory wall. The window is not a capacity choice: the engine's own
+    # reserve model (one max_model_len request) needs a large L for this
+    # model's per-sequence KDA state, and below ~92k the boot REFUSES with
+    # "needed ... larger than the available KV cache memory". NBATCH pins the
+    # prefill chunk that was measured.
+    grep -q '^FRANK_PRIMARY_MAXLEN=131072$' "$ENV_FILE"
+    grep -q '^FRANK_PRIMARY_SEQS=2$' "$ENV_FILE"
     grep -q '^FRANK_PRIMARY_UTIL=0.98$' "$ENV_FILE"
+    grep -q '^FRANK_PRIMARY_NBATCH=2048$' "$ENV_FILE"
+    grep -q '^FRANK_PRIMARY_MODEL_SHARDS=34$' "$ENV_FILE"
 }
 
 @test "the serve script wires the engine flags from the env" {
-    grep -q -- '--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE"' "$SERVE"
-    grep -q -- '--kv-cache-memory-bytes "$FRANK_PRIMARY_KV_BYTES"' "$SERVE"
     grep -q -- '--max-num-seqs "$FRANK_PRIMARY_SEQS"' "$SERVE"
     grep -q -- '--max-model-len "$FRANK_PRIMARY_MAXLEN"' "$SERVE"
+    grep -q -- '--max-num-batched-tokens "$FRANK_PRIMARY_NBATCH"' "$SERVE"
     grep -q -- '--tensor-parallel-size 2' "$SERVE"
-    grep -q 'num_speculative_tokens' "$SERVE"
     grep -q -- '--enable-prefix-caching' "$SERVE"
     grep -q -- '--trust-remote-code' "$SERVE"
 }
 
+@test "the endpoint advertises the GLM tool dialect" {
+    # Without --enable-auto-tool-choice the engine rejects tool_choice "auto"
+    # with a 400 ("requires --enable-auto-tool-choice and --tool-call-parser"),
+    # which breaks every tool-using agent on the endpoint.
+    grep -q -- '--enable-auto-tool-choice' "$SERVE"
+    grep -q -- '--tool-call-parser "$FRANK_PRIMARY_TOOLPARSER"' "$SERVE"
+    grep -q -- '--reasoning-parser "$FRANK_PRIMARY_REASONINGPARSER"' "$SERVE"
+    grep -q '^FRANK_PRIMARY_TOOLPARSER=glm47$' "$ENV_FILE"
+    grep -q '^FRANK_PRIMARY_REASONINGPARSER=glm47$' "$ENV_FILE"
+}
+
+@test "expert offload is ON — the payload cannot fit VRAM without it" {
+    # 170 GiB of int4 experts against 64 GiB of VRAM. Without the offload flag
+    # the engine has no home for the expert tensors at all; the arena
+    # (14.51 GiB = 2,383 slots) plus 73.64 GiB/rank of pinned host rows is the
+    # only shape that boots. --expert-precise-memory is the qualified sizing.
+    grep -q -- '--enable-expert-offload' "$SERVE"
+    grep -q -- '--expert-precise-memory' "$SERVE"
+}
+
+@test "MTP spec decode is absent BY MEASUREMENT, not by omission" {
+    # 2026-09-30: the MTP head loads and accepts well (mean accepted length
+    # 3.76-4.90), but decode was unchanged (9.6-11.2 vs 10.0 tok/s) while
+    # prefill collapsed 918 -> 255 tok/s — the verification forward gathers the
+    # union of the batch's experts, so spec decode does not amortize a
+    # residency-bound gather. It also costs 2.6 GiB of arena.
+    ! grep -q -- '--speculative-config' "$SERVE"
+    ! grep -q 'speculative' "$ENV_FILE"
+}
+
+@test "fp8 KV is absent — it would disable the only MLA decode path on gfx12" {
+    # _use_rocm_sparse_triton() in the ROCM_AITER_MLA_SPARSE backend requires a
+    # non-fp8 KV cache, and that rope-free Triton path is the only MLA decode
+    # available on gfx12: aiter's MLA decode op is registered under
+    # @if_aiter_supported, gated on CDNA>2. With fp8 KV the MTP draft step dies
+    # with "_OpNamespace 'vllm' object has no attribute
+    # 'rocm_aiter_mla_decode_fwd'". KV dtype therefore stays auto/bf16.
+    ! grep -q -- '--kv-cache-dtype' "$SERVE"
+    ! grep -q '^FRANK_PRIMARY_KV_DTYPE=' "$ENV_FILE"
+}
+
 @test "expert prefetch is absent BY LAW, not by omission" {
     # Telemetry (2026-09-29): the prefetcher served 13,369 of 36,720
     # prefetched slabs (36% precision) — roughly half the gather traffic was
@@ -149,10 +190,10 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
 @test "the model id matches the payload's served alias wiring" {
     # The payload dir and its shard count are the fetch script's output
     # layout; the fetch script must write where the env points.
-    grep -q '^FRANK_PRIMARY_MODEL_DIR=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8$' "$ENV_FILE"
-    grep -q '^FRANK_PRIMARY_MODEL_SHARDS=131$' "$ENV_FILE"
-    grep -q 'Qwen3.8-Flash-Next-Uncensored-FP8' "$DOWNLOAD"
-    grep -q 'orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8' "$DOWNLOAD"
+    grep -q '^FRANK_PRIMARY_MODEL_DIR=/home/dank/models/GLM-5.3-Flash-W4A16-AutoRound$' "$ENV_FILE"
+    grep -q '^FRANK_PRIMARY_MODEL_SHARDS=34$' "$ENV_FILE"
+    grep -q 'GLM-5.3-Flash-W4A16-AutoRound' "$DOWNLOAD"
+    grep -q 'Intel/GLM-5.3-Flash-W4A16-AutoRound' "$DOWNLOAD"
 }
 
 @test "no bare HIP-reserved env var is defined" {
@@ -168,12 +209,13 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
     # storage or the payload is missing/wrong-sharded — then clears stale
     # containers (including the legacy identities), waits out the port,
     # relinks/enables/restarts the root unit and health-gates it.
-    grep -q 'LLM_PRIMARY_CONTAINER=frank-primary-qwen' "$REMOTE_DEPLOY"
+    grep -q 'LLM_PRIMARY_CONTAINER=frank-primary-glm' "$REMOTE_DEPLOY"
     grep -q 'FRANK_PRIMARY_IMAGE' "$REMOTE_DEPLOY"
     grep -q "podman images --format '{{.RepoDigests}}'" "$REMOTE_DEPLOY"
     grep -q 'FRANK_PRIMARY_MODEL_SHARDS' "$REMOTE_DEPLOY"
     grep -q 'config.json' "$REMOTE_DEPLOY"
     grep -q 'qwen-fp8' "$REMOTE_DEPLOY"
+    grep -q 'glm-probe' "$REMOTE_DEPLOY"
     grep -q 'wait_port_free 7770' "$REMOTE_DEPLOY"
     grep -q 'relink_unit frank-primary' "$REMOTE_DEPLOY"
     grep -q 'systemctl enable frank-primary.service' "$REMOTE_DEPLOY"