frank-primary cutover: GLM-5.3-Flash W4A16 int4 (Intel AutoRound)

Payload: Intel/GLM-5.3-Flash-W4A16-AutoRound (AutoRound symmetric int4 group
128, auto_round:auto_gptq, 34 shards + 2 extras, 113,074 tensors, ~170 GiB).
It is the only W4A16 checkpoint this fork can serve with expert offload: its
quant_method "auto-round" is overridden to "inc", which routes the MoE to
libr4d's grouped W4A16 GEMM (R4dW4a16MoEMethod). A compressed-tensors W4A16
build of the same model resolves to TritonWNA16Experts, which has no offload
recipe, and refuses to serve at all (170 GiB of experts cannot fit 64 GiB of
VRAM, so offload is mandatory).

Shape: tcclaviger RDNA4 vLLM fork (same digest-pinned image as the 2026-09-29
qwen cutover), TP=2, 131,072 ctx x 2 seqs, bf16 KV (fp8 is ILLEGAL here — it
disables the sparse MLA backend's rope-free Triton path, the only MLA decode
gfx12 has), util 0.98, NBATCH 2048, --enable-expert-offload
--expert-precise-memory, prefix caching, glm47 tool + reasoning parsers,
MTP and expert prefetch absent by measurement.

Measured: decode 9.9-10.0 tok/s, prefill 918 tok/s (warm), arena 14.51 GiB =
2,383 slots/rank over 12,096 host-backed experts (73.64 GiB pinned/rank).
THE 35 tok/s TARGET IS NOT MET and the shortfall is measured: live telemetry
(R4D_LRU_TELEMETRY=1) gives a 0.53-0.66 per-layer hit rate, a step moves
0.82 GiB of misses in ~100 ms = 8.2 GB/s effective against a link measured at
22.74 GB/s — the same structural gather-path gap the qwen campaign found.
Growing the arena 13.08 -> 14.51 GiB changed decode by nothing.

Rejected by measurement: MTP4 (head loads, acceptance 3.76-4.90, but decode
unchanged and prefill collapses 918 -> 255 — a verification forward gathers the
union of the batch's experts), NBATCH 8192 (identical prefill).

Also: scripts/patches/gpu_worker-kv-cap.patch (fork-local fix for a KV-reserve
model that under-counts GLM's per-sequence KDA state; not needed by this shape),
runbook, AGENTS.md row + section, both indexes, work log.

Rollback: 2339fae (qwen FP8 + MTP4).

scripts/frank-primary.service

@@ -1,26 +1,24 @@
-# frank-primary.service — tcclaviger RDNA4 vLLM endpoint (orcarouter
-# Qwen3.8-Flash-Next-Uncensored-FP8, MTP4 spec decode, KV q8/q8, 600k total
-# context over 3 sessions, TP=2, tiered experts, 131-shard payload) on the
-# 2x R9700 pair, dankserver :7770. 2026-09-29 cutover from the R9V radiance
-# fork (git tag frank-primary-r9v-q8-20260929 is the pre-cutover state; its
-# launcher/shim/setup-state machinery is retired and the state dirs stay on
-# disk as rollback).
+# frank-primary.service — tcclaviger RDNA4 vLLM endpoint
+# (Intel/GLM-5.3-Flash-W4A16-AutoRound int4, 131,072 x 2 seqs, bf16 KV, TP=2,
+# expert offload to pinned host RAM, 34-shard payload) on the 2x R9700 pair,
+# dankserver :7770. 2026-09-30 cutover from orcarouter
+# Qwen3.8-Flash-Next-Uncensored-FP8 + MTP4; commit 2339fae is the pre-cutover
+# state and the rollback.
 #
 # Same port, same served alias; the harness models.yml id is unchanged.
 # The serve script drives `podman run` directly (the image entrypoint execs
 # `vllm serve`) and then waits on the container, so a crashed engine is a
 # nonzero ExecStart and Restart=on-failure is real.
 #
-# WHY A ROOT UNIT (not a rootless quadlet): the tiered-expert shape pins tens
-# of GiB of host RAM (the expert host slabs, ~20.5 GiB per rank of pinned
-# anonymous memory). Pinned pages are charged against RLIMIT_MEMLOCK; dank's
+# WHY A ROOT UNIT (not a rootless quadlet): the expert-offload shape pins
+# 73.64 GiB per rank of host RAM (the expert host rows). Pinned pages are charged against RLIMIT_MEMLOCK; dank's
 # login session on this host carries 8 MiB and a user quadlet is capped by the
 # user manager's own hard limit — raising that means restarting
 # user@1000.service and bouncing every user container on the host. A root unit
 # grants LimitMEMLOCK=infinity to the dank process via PID1, and the image
 # stays in dank's rootless podman storage.
 [Unit]
-Description=frank-primary — tcclaviger vLLM Qwen3.8-Flash-Next-Uncensored-FP8 on 2x R9700
+Description=frank-primary — tcclaviger vLLM GLM-5.3-Flash int4 W4A16 on 2x R9700
 After=network-online.target
 Wants=network-online.target
 
@@ -35,9 +33,9 @@ Environment="FRANK_PRIMARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-pr
 ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-primary.sh
 # systemd stop: wait alone cannot stop the container child; stop it
 # explicitly. vLLM 0.26 drains gracefully on SIGTERM.
-ExecStop=/usr/bin/podman stop -t 120 frank-primary-qwen
-# Boot cost: FP8 payload page-in plus torch compile/graph capture —
-# ~15 min cold (measured 2026-09-29), faster warm.
+ExecStop=/usr/bin/podman stop -t 120 frank-primary-glm
+# Boot cost: int4 payload page-in plus torch compile/graph capture —
+# ~7 min measured 2026-09-30 on a warm page cache, longer cold.
 TimeoutStartSec=1800
 Restart=on-failure
 RestartSec=10