frank-primary: DFlash2 load attempt + fp8 verdict + compile blocker narrowed

DFlash2: fetched incoai/GLM-5.3-Flash-DFlash2 (2,342,169,800 B, sha256 matches the
Hub LFS oid) and tried --speculative-config {"method":"dflash","model":"/draft"}.
Config and drafter are accepted; the worker dies with 'Model does not support
EAGLE3 interface' - so the only gap is the target-side aux-hidden-state contract
(set_aux_hidden_state_layers + per-layer capture + the two-value forward return,
as deepseek_v2.py demonstrates twice in this image). Recorded the exact interface,
the GLM-specific unknown (what the single-stream state entering layer k means for
an mHC multi-stream trunk) and the honest expectation: no speedup on a
token-proportional pointwise bound.

fp8 KV: recorded the deciding measurement - the fork's KV reserve model is
dtype-blind (the fp8 boot printed the identical 1760 MiB / 13600 B-per-token plan),
so fp8 cannot buy capacity even if the gfx12 MLA path were rewritten. Verdict: do
not spend effort there.

Compile blocker narrowed from 'a latent GLM bug' to dynamo's name map: the
KeyError is thrown by torch/_dynamo/output_graph.py's nn_modules lookup (:3000,
registered :1259/:1384, _modules[..] prefixes stripped :1225), i.e. the compiler
cannot resolve a submodule by name while tracing the decoder layer. Next step is
TORCHDYNAMO_VERBOSE=1, with the note that the scratch launcher resets ENVARGS.

tests/run.sh: 362 seams pass.

serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md

@@ -175,18 +175,30 @@ What the boots established, in order (each one ~7 min):
    `RuntimeError: Graph breaks are not supported with aot compile`, because this
    build runs AOT compile by default and AOT refuses breaks.
 3. **01+02+03 with `VLLM_COMPILE_FULLGRAPH=0 VLLM_USE_AOT_COMPILE=0` traces the
-   trunk successfully** (`Dynamo bytecode transform time: 1.08 s`) and then dies in
-   `profile_run` with `KeyError: 'in_proj_qkvbfg_a'` at
-   `glm5next/common/model.py:456` (the KDA layer's merged projection,
-   `kda.py:217`) — a dict/registry lookup keyed by attribute name that the eager
-   path never reaches, i.e. **a latent bug the compile exposes, not a dynamo
-   limit**.
+   trunk successfully** (`Dynamo bytecode transform time: 1.08 s`) and then dies on
+   the decode forward with `KeyError: 'in_proj_qkvbfg_a'` (`Glm5NextModel.forward`
+   line 748 calling the decoder layer at line 456). `in_proj_qkvbfg_a` is the KDA
+   layer's merged column-parallel projection (`kda.py:217`, called at :407) — a real
+   submodule attribute name, also present in the `load_weights` shard table
+   (`model.py:780-785`). Reading the full traceback (the reported line is inside
+   dynamo-rewritten code) traces the throw to dynamo's own name→module map:
+   `torch/_dynamo/output_graph.py` registers into `nn_modules` at :1259/:1384,
+   looks up with `self.nn_modules[node.target]` at :3000 (a bare-name `KeyError`
+   when absent), and documents stripping `_buffers[..]/_parameters[..]/_modules[..]`
+   from those names at :1225. So the compiler is failing to **resolve a submodule by
+   name** while tracing the GLM decoder layer; whether the root cause is the fork's
+   module tree or dynamo's registration is not yet settled. The next step is the one
+   torch itself suggests in that error — `TORCHDYNAMO_VERBOSE=1` for the internal
+   stack trace (and note the launcher resets `ENVARGS`, so it must be added inside
+   `serve_glm_autoround.sh`'s COMPILE_PATCH branch rather than in the outer
+   environment).
 
-That last row is the state of the work: the remaining blocker is a real bug in the
-fork's mHC/KDA wiring, and it must be fixed (plus whatever follows it) before the
-changelist can boot. Two verified facts make the payoff concrete: `suppress_errors`
-does **not** cover graph breaks (only `fullgraph=False` does), and the GLM trunk can
-never satisfy `fullgraph` because its indexer metadata syncs to host
+That last row is the state of the work: the changelist is one unresolved
+submodule-name lookup away from booting, and it must be cleared (plus whatever
+follows it) before the compile payoff can be measured. Two verified facts make the
+rest of the shape concrete: `suppress_errors` does **not** cover graph breaks (only
+`fullgraph=False` does), and the GLM trunk can never satisfy `fullgraph` because
+its indexer metadata syncs to host
 (`glm5next/amd/sparse_indexer.py:263,419`: `int(...max().item())`).
 
 ### Two structural constraints on any further fork work
@@ -221,10 +233,48 @@ layer indices, the same interface `deepseek_v2.py` (also MLA+MoE) implements —
 capture at `idx in self.aux_hidden_state_layers` in the layer loop. `glm5next`
 implements neither.
 
-Expected benefit is **not** assumed: DFlash2 amortizes a draft block (8 tokens) over
-one verification pass, and per the profile above a verification pass costs pointwise
-work proportional to tokens — the same reason MTP4 was flat. Measure it; do not
-budget on it.
+**The load was tried, and the gap is exactly one interface.** Payload fetched to
+`/home/dank/models/GLM-5.3-Flash-DFlash2` and hash-verified (`model.safetensors`
+2,342,169,800 B, sha256 `b038e1d9…` = the Hub's LFS oid). Booting the engine with
+`--speculative-config '{"method":"dflash","model":"/draft","num_speculative_tokens":8}'`
+accepts the config and the drafter and dies at worker init with:
+
+```
+RuntimeError: Model does not support EAGLE3 interface
+```
+
+So nothing about the drafter, the method name, or the payload is wrong — the GLM
+trunk simply does not implement the aux-hidden-state contract the dflash runner
+requires. The reference is in this same image, twice: `deepseek_v2.py`
+(`set_aux_hidden_state_layers` / `get_eagle3_aux_hidden_state_layers` at :1948-1951,
+capture at :1554 and :1579) and the fork's own `qwen4_exp` trunk. The contract is
+small:
+
+- `self.aux_hidden_state_layers: tuple[int, ...]` on the trunk, default `()`, set
+  by the wrapper's `set_aux_hidden_state_layers()`;
+- in the layer loop, `if idx in self.aux_hidden_state_layers: aux_hidden_states.append(<single-stream state entering layer idx>)`
+  (DeepSeek appends `hidden_states + residual`, with a TP all-gather fixup when the
+  rank holds fewer tokens than `positions`);
+- `Glm5NextModel.forward` returns `(hidden_states, aux_hidden_states)` when the
+  list is non-empty;
+- `get_eagle3_aux_hidden_state_layers()` supplies a default triple when the drafter
+  config omits `target_layer_ids` (this drafter provides `[5, 14, 24, 33, 42]`, so
+  the runner derives trunk indices `[6, 15, 25, 34, 43]`).
+
+**The GLM-specific unknown** is what "the single-stream state entering layer k"
+means for a trunk whose layers carry mHC multi-stream state: DeepSeek's
+`(hidden_states, residual)` pair has no counterpart here — GLM's decoder layer
+returns `(hidden_states, residual, post, comb)` with `hc_expand`/`hc_pre` mixing
+`hc_mult` streams, so the plausible answer is the pre-mHC single-stream input `x`
+(4096-wide, matching the drafter's `hidden_size`), not a post-layer value. That is
+a semantic choice to validate by acceptance rate, not by inspection.
+
+**Budget no speedup.** Per the profile above, a verification pass pays pointwise
+cost proportional to its token count — the same reason MTP4 measured flat — and
+MTP4's acceptance was already excellent (3.76-4.90). The port is worth doing as
+*enabling work* (it is the prerequisite for any spec decode on this endpoint, and
+it is small), not as a performance fix, and the honest expectation on this shape is
+flat decode with a prefill regression like MTP's until the pointwise mass is fused.
 
 Note also that the KV-cap patch's predecessor arithmetic in
 [[dankserver-qwen38-fp8-expert-offload-tuning]] should be read the same way: its
@@ -275,17 +325,32 @@ newest GLM dialect the fork ships (`glm45`/`glm47` exist for both).
 
 ## Fork traps found (for the engine authors)
 
-1. **`--kv-cache-dtype fp8` is illegal on gfx12 with this model.** The
-   `ROCM_AITER_MLA_SPARSE` backend's `_use_rocm_sparse_triton()` requires a
-   non-fp8 KV cache, and that rope-free Triton path is the **only** MLA decode
-   available on gfx12: aiter's MLA decode op is registered under
-   `@if_aiter_supported`, which is gated on **CDNA > 2**
+1. **`--kv-cache-dtype fp8` is illegal on gfx12 with this model — and it buys
+   nothing even if fixed.** The `ROCM_AITER_MLA_SPARSE` backend's
+   `_use_rocm_sparse_triton()` requires a non-fp8 KV cache, and that rope-free
+   Triton path is the **only** MLA decode available on gfx12: aiter's MLA decode
+   op is registered under `@if_aiter_supported`, which is gated on **CDNA > 2**
    (`is_aiter_found_and_supported()` → `get_cdna_version() > 2`). With fp8 KV
    the MTP draft step demands an op that cannot exist here:
    `RuntimeError: Worker failed with error ''_OpNamespace' 'vllm' object has no
    attribute 'rocm_aiter_mla_decode_fwd''`. KV dtype therefore stays auto/bf16.
    (`VLLM_ROCM_USE_AITER_MLA=1` does **not** help — the umbrella gate is the
    CDNA check, not the env var.)
+   - **The capacity payoff is measured at zero.** The point of fp8 KV is fitting
+     more tokens in the same reserve, and this fork's reserve model is
+     **dtype-blind**: the fp8 boot printed the *identical* plan line as bf16 —
+     `KV reserve 1.76 GiB (13600 B/token x 131072 + mamba pages)`. The reserve is
+     dominated by the per-sequence KDA state (34 linear-attention layers) and the
+     DSA indexer's pools, not by the MLA KV bytes those 13,600 B/token barely
+     cover — so halving the KV dtype cannot move it.
+   - **What a fix would take, if it were ever wanted**: a new fp8 path in the
+     rope-free Triton kernels (`rocm_sparse_attn_prefill` / `rocm_sparse_attn_decode`
+     in `v1/attention/ops/rocm_aiter_mla_sparse.py`) that dequantizes pages
+     in-kernel, plus loosening `_use_rocm_sparse_triton()`. The fork's
+     `_r4d_fp8_paged_mqa_logits` shows the author already has an fp8 MQA-logits
+     path, but the sparse-MLA backend is not wired to it. Verdict 2026-09-30:
+     **do not spend effort here** — the payoff is capacity, and the reserve is
+     dtype-blind.
 2. **The plan's KV reserve under-models this model, and the cap enforces the
    mistake.** The plan charges 13,600 B/token + a small mamba term, but GLM
    carries a per-sequence KDA state (34 linear-attention layers) and, under MTP,
@@ -322,6 +387,16 @@ podman logs frank-primary-glm 2>&1 | grep "expert_lru.py"
 
 # boot: ~7 min warm page cache, longer cold; health returns 503 while loading
 curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:7770/health
+
+# experiment windows (production stopped + `systemctl mask --runtime`, which
+# self-clears at reboot) ran from the host-side scratch launcher
+# /home/dank/serve_glm_autoround.sh: env knobs MAXLEN SEQS NBATCH UTIL KVBYTES
+# KVDTYPE SPEC SPECJSON_FILE TOOLPARSER REASONINGPARSER TELEM GCHUNKS GLANES
+# PROFDIR GLM_PATCH_MOUNT COMPILE_PATCH GATHER_PATCH DRAFT_MOUNT ENVARGS.
+# DRAFT_MOUNT=1 mounts /home/dank/models/GLM-5.3-Flash-DFlash2 at /draft;
+# SPECJSON_FILE passes a whole --speculative-config JSON (JSON with no spaces,
+# word-split by the launcher's unquoted $EXTRA).
+bash /home/dank/serve_glm_autoround.sh
 ```
 
 Boot gate: the engine refuses a legal-looking shape in two measured ways — the

serverAppConfigs-docs/log.md

@@ -66,6 +66,21 @@
     capture (`set_aux_hidden_state_layers` / per-layer capture, as `deepseek_v2.py`
     demonstrates). No speedup is budgeted: a verification pass pays pointwise cost
     proportional to its token count.
+  - **DFlash2 tried, and the gap is exactly that one interface.** Payload fetched
+    and hash-verified (`/home/dank/models/GLM-5.3-Flash-DFlash2`, 2,342,169,800 B,
+    sha256 `b038e1d9…` = the Hub's LFS oid). Booting with
+    `--speculative-config '{"method":"dflash","model":"/draft","num_speculative_tokens":8}'`
+    accepts the config and the drafter and dies at worker init with
+    `RuntimeError: Model does not support EAGLE3 interface` — nothing about the
+    drafter, the method or the payload is wrong; the trunk simply does not
+    implement the aux-hidden-state contract.
+  - **Compile blocker narrowed to dynamo's name map.** The `KeyError:
+    'in_proj_qkvbfg_a'` (see above) is thrown from `torch/_dynamo/output_graph.py`'s
+    `nn_modules` name lookup (:3000, registered at :1259/:1384, with `_modules[..]`
+    prefixes stripped at :1225) — i.e. the compiler cannot resolve a submodule by
+    name while tracing the GLM decoder layer. Next step is `TORCHDYNAMO_VERBOSE=1`
+    (torch names it in the error), noting the scratch launcher resets `ENVARGS` so
+    it must be set inside the script's COMPILE_PATCH branch.
   - Measured and rejected: **MTP4** (the head loads, arch resolves to
     `Glm5NextMTPModel`, acceptance is excellent at mean length 3.76-4.90 — but
     decode was unchanged (9.6-11.2 vs 10.0) while prefill COLLAPSED 918 -> 255.