frank-primary: profile the GLM decode step and stage the compile changelist

A torch-profiler kernel-time window (496 decode tokens, rank0 trace parsed for
device-activity time) on the deployed GLM-5.3-Flash shape falsifies this page's
own "the expert gather is the bound" explanation:

  46.18 s of kernel time / 496 tokens = ~93 ms per token (actual step ~104 ms)
   34.8 %  vectorized_elementwise_kernel      708,500 launches (1,428/token)
   10.0 %  r4d_gemm_moe_w4a16_nt_b16_m64      (the MoE GEMM)
    6.7 %  elementwise_kernel_manual_unroll
    6.4 %  wvSplitK_hf_sml_                    (split-K GEMM)
    8.9 %  router: sbtopk + warpMergeSortTopK + bitonicSortKVInPlace
    6.5 %  r4d_mhc_*                           (hyper-connections)
    1.05 % r4d_moe_lru_gather_kernel           <-- the retracted suspect

The earlier arithmetic was circular (it assumed the gather consumed the whole
step). The real bound is unfused pointwise glue: the fork compiles its own Qwen
trunk (support_torch_compile in models/qwen4_exp/amd/model.py) and not its GLM
trunk, so the engine boots with "provides neither a compiled submodule nor
breakable CUDA graphs. Overriding to FULL_DECODE_ONLY." That also explains why
MTP4 measured flat — pointwise cost scales with the token count, so spec decode
amortizes nothing here — and it retires arena/hit-rate tuning (the gather is 1 %).

Staged as documented patch artifacts (not wired into any serve script; the
engine does not yet boot with them) — scripts/patches/glm53-compile-0{1,2,3}*.patch:

  -01 models/glm5next/common/model.py    decorate Glm5NextModel (R4D_GLM_COMPILE)
  -02 model_executor/kernels/r4d_lib.py  freeze 10 capability probes that do
      os.path.exists/.so lookups inside the traced forward. functools.lru_cache
      does NOT protect them: dynamo re-traces the wrapped body even when warm
      (verified in-image); a plain module-global read compiles.
  -03 compilation/wrapper.py             make fullgraph=True opt-out, because
      suppress_errors does NOT cover graph breaks and this trunk can never
      satisfy fullgraph (indexer metadata syncs to host — .item() at
      glm5next/amd/sparse_indexer.py:263,419).

With all three the trunk traces (Dynamo bytecode transform time 1.08 s) and dies
on KeyError: in_proj_qkvbfg_a in the mHC/KDA path — a latent bug the compile
exposes, and the remaining blocker.

Also records two structural constraints: no new HIP kernels are possible (r4d.so
ships without sources), which binary-blocks the fused router at GLM's 288
experts; and DFlash2 for GLM is a TARGET-side port, not a drafter port
(incoai/GLM-5.3-Flash-DFlash2 is architectures ["DFlash2DraftModel"] +
model_type qwen3, already mapped at registry.py:636; glm5next lacks only the
aux-hidden-state plumbing deepseek_v2.py demonstrates).

Runbook frontmatter, intro, mermaid and wall section, AGENTS.md and the work log
corrected to match the measurement. tests/run.sh: 362 seams pass.

AGENTS.md

@@ -24,7 +24,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
-| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `Intel/GLM-5.3-Flash-W4A16-AutoRound` — AutoRound symmetric int4 g128 (170 GiB, 34 shards, 113,074 tensors), TP=2, expert offload to 73.64 GiB/rank of pinned host rows behind a 14.51 GiB / 2,383-slot VRAM LRU arena, 131072 ctx × 2 seqs, bf16 KV, no MTP, no prefetch; measured 9.9-10.0 tok/s decode / 918 tok/s prefill | 7770 |
+| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `Intel/GLM-5.3-Flash-W4A16-AutoRound` — AutoRound symmetric int4 g128 (170 GiB, 34 shards, 113,074 tensors), TP=2, expert offload to 73.64 GiB/rank of pinned host rows behind a 14.51 GiB / 2,383-slot VRAM LRU arena, 131072 ctx × 2 seqs, bf16 KV, no MTP, no prefetch; measured 9.9-10.0 tok/s decode / 918 tok/s prefill, bounded by unfused pointwise glue (the expert gather is 1.05 % of device time; the trunk is uncompiled) | 7770 |
 | Dankclaw | `frank-large` | gufo `gufo-org/gufo` v0.1.1, the Strix Halo engine, BUILT FROM SOURCE (root systemd unit; links the host ROCm 7.2.4) | Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (27B dense, ABLITERATED/uncensored, 29.3 GiB, SWIFT OPEN LICENSE 1.0) with the z-lab DFlash2 Q4_K_M drafter (adaptive block policy, cap 7) on the gfx1151 iGPU (Strix Halo LPDDR, 114 GiB GTT), 262144 ctx single session, greedy, gufo's compiled-in Qwen renderer (NO Qwen-Sharp), text only | 7770 |
 | Dankclaw | `frank-small` | mainline llama.cpp master 37b53fd (root systemd unit) | HauhauCS Gemma4-12B-QAT-Uncensored-Balanced Q4_K_M (BF16 mmproj + MTP assistant sidecar, `draft-mtp,ngram-mod`) on the RTX 3090, 1048576 pool / 4 slots of 262,144, q8_0 KV, the fleet's fastest vision endpoint | 7771 |
 
@@ -445,14 +445,51 @@ serve. Built shape: 14.51 GiB arena = **2,383 slots × 6.234 MiB per rank** over
 12,096 host-backed experts = **73.64 GiB pinned host rows per rank**.
 
 **Measured 2026-09-30: decode 9.9–10.0 tok/s, prefill 918 tok/s** (warm, 2.6k
-prompt), KV cache 138,561 tokens. **The 35 tok/s target is NOT met, and this is
-a measured wall rather than a tuning gap:** each decode step must move 2.045 GiB
-of routed experts per rank (42 layers × 8 experts × 6.234 MiB), live telemetry
-(`R4D_LRU_TELEMETRY=1`) puts the expert-cache hit rate at **0.53–0.66**, and the
-fork's gather read path delivers **~8.2 GB/s against a link measured at
-22.74 GB/s** — the same structural gap the qwen campaign found. Growing the
-arena 13.08 → 14.51 GiB changed decode by *nothing* (10.03 → 9.88 tok/s), so
-residency stopped being the binding term; 0.89 hit rate would be needed for 35.
+prompt), KV cache 138,561 tokens. **The 35 tok/s target is NOT met. The first
+explanation for that — "the expert gather is the bound at ~8.2 GB/s" — was
+FALSIFIED by a torch-profiler kernel-time window** (496 decode tokens, rank0
+trace parsed for device-activity time): 46.18 s of kernel time = ~93 ms per
+token, of which **44 % is pointwise glue** — `vectorized_elementwise_kernel`
+alone is 16.07 s over **708,500 launches (1,428 per token)**, plus 3.11 s of
+`elementwise_kernel_manual_unroll` — against **1.05 % for
+`r4d_moe_lru_gather_kernel`**. The MoE GEMM is 10 %, the split-K GEMM 6 %, the
+router/top-k 9 %, the mHC kernels 6.5 %. The real wall is that **the fork
+compiles its own Qwen trunk (`@support_torch_compile` in
+`models/qwen4_exp/amd/model.py`) and not its GLM trunk**, so every pointwise op
+is a separate eager kernel. Two consequences: spec decode **cannot** help this
+shape (pointwise cost scales with the token count, which is exactly why MTP4
+measured flat), and neither arena size nor hit rate is a lever (the gather is
+1 %).
+
+**Fork work is staged, not deployed.** `scripts/patches/glm53-compile-0{1,2,3}*.patch`
+are the compile-enablement changelist (decorate the trunk; freeze the ten libr4d
+capability probes that do `os.path.exists`/`.so` lookups *inside* the traced
+forward; make `fullgraph=True` opt-out). Boots established that
+`functools.lru_cache` does **not** protect those probes (dynamo re-traces the
+wrapped body even warm), that `suppress_errors` does **not** cover graph breaks
+(only `fullgraph=False` does), and that the GLM trunk can never satisfy
+`fullgraph` (its indexer metadata syncs to host at
+`glm5next/amd/sparse_indexer.py:263,419`). With all three the trunk **traces**
+(`Dynamo bytecode transform time: 1.08 s`) and then dies on
+`KeyError: 'in_proj_qkvbfg_a'` in the mHC/KDA path — a latent bug the compile
+exposes, and the remaining blocker.
+
+**No new HIP kernels are possible** — the image ships `r4d.so` as a binary with
+no sources. The 9 % router is therefore **binary-blocked**: its fused path is
+gated `num_experts in {128, 256, 512}` (`router/r4d_route_router.py`) and only
+`r4d_moe_route_kernel<128|256|512>` instantiations exist, while GLM has **288**
+experts. Only Python/config/inductor changes (or runtime-JIT Triton) are
+available on this fork.
+
+**DFlash2 for GLM is a target-side port, not a drafter port.**
+`incoai/GLM-5.3-Flash-DFlash2` declares `architectures: ["DFlash2DraftModel"]`
+and `model_type: "qwen3"` (5 dense layers, sliding attention, block 8,
+`target_layer_ids [5, 14, 24, 33, 42]` of 45), and the fork **already maps it**
+(`registry.py:636` → `qwen3_dflash2.DFlash2Qwen3ForCausalLM`; spec method
+`"dflash"`). What is missing is the target side: `glm5next` implements neither
+`set_aux_hidden_state_layers()` nor the per-layer aux capture that
+`deepseek_v2.py` (also MLA+MoE) demonstrates. Budget no speedup: per the profile
+above a verification pass pays pointwise cost proportional to its token count.
 
 **MTP works here and is still rejected.** The head loads
 (`Resolved architecture: Glm5NextMTPModel`) and accepts well (mean accepted