frank-primary: DFlash2 load attempt + fp8 verdict + compile blocker narrowed

DFlash2: fetched incoai/GLM-5.3-Flash-DFlash2 (2,342,169,800 B, sha256 matches the
Hub LFS oid) and tried --speculative-config {"method":"dflash","model":"/draft"}.
Config and drafter are accepted; the worker dies with 'Model does not support
EAGLE3 interface' - so the only gap is the target-side aux-hidden-state contract
(set_aux_hidden_state_layers + per-layer capture + the two-value forward return,
as deepseek_v2.py demonstrates twice in this image). Recorded the exact interface,
the GLM-specific unknown (what the single-stream state entering layer k means for
an mHC multi-stream trunk) and the honest expectation: no speedup on a
token-proportional pointwise bound.

fp8 KV: recorded the deciding measurement - the fork's KV reserve model is
dtype-blind (the fp8 boot printed the identical 1760 MiB / 13600 B-per-token plan),
so fp8 cannot buy capacity even if the gfx12 MLA path were rewritten. Verdict: do
not spend effort there.

Compile blocker narrowed from 'a latent GLM bug' to dynamo's name map: the
KeyError is thrown by torch/_dynamo/output_graph.py's nn_modules lookup (:3000,
registered :1259/:1384, _modules[..] prefixes stripped :1225), i.e. the compiler
cannot resolve a submodule by name while tracing the decoder layer. Next step is
TORCHDYNAMO_VERBOSE=1, with the note that the scratch launcher resets ENVARGS.

tests/run.sh: 362 seams pass.
This commit is contained in:
Comrade Chief Designer
2026-09-30 12:49:26 -07:00
parent bd8c72ff47
commit 1c18b9c808
3 changed files with 133 additions and 28 deletions
@@ -38,18 +38,33 @@
#
# NEXT BLOCKER — NOT FIXED HERE. With 01+02+03 and
# R4D_GLM_COMPILE=1 VLLM_COMPILE_FULLGRAPH=0 VLLM_USE_AOT_COMPILE=0 the trunk traces
# ("Dynamo bytecode transform time: 1.08 s") and then the worker dies during profile_run:
# ("Dynamo bytecode transform time: 1.08 s") and then the worker dies during profile_run
# on the decode forward:
#
# KeyError: 'in_proj_qkvbfg_a'
# vllm/models/glm5next/common/model.py:456 in Glm5NextDecoderLayer.forward
# model.py:748 Glm5NextModel.forward hidden_states, residual, post, comb = layer(...)
# model.py:456 Glm5NextDecoderLayer.forward
#
# `in_proj_qkvbfg_a` is the KDA layer's merged column-parallel projection
# (models/glm5next/common/kda.py:217, called at :407). The key is looked up in a
# dict/mapping that the EAGER path never reaches, so compiling the trunk exposes a
# latent bug in the mHC/KDA wiring rather than a dynamo limitation. Fix that and
# whatever follows before this changelist can boot — then re-measure: the prize is the
# 44 % pointwise mass (see 01), and the frozen-probe log line is the signal that 02 is
# live.
# (models/glm5next/common/kda.py:217, called at :407) — and it is also a name in the
# load_weights shard table (model.py:780-785), i.e. a real submodule attribute name,
# not an arbitrary string.
#
# WHERE IT IS THROWN (established 2026-09-30 by reading the full traceback, since the
# report line is inside dynamo-rewritten code): torch/_dynamo keeps a name -> module map
# and looks it up by name — output_graph.py:1259 / :1384 register into `nn_modules`,
# :3000 does `self.nn_modules[node.target]` (a bare-name KeyError when absent), and
# :1225 documents stripping "_buffers[..]/_parameters[..]/_modules[..]" from those
# names. So the compiler is failing to RESOLVE A SUBMODULE BY NAME while tracing the
# GLM decoder layer. Whether the root cause is the fork's module tree (something
# dynamo cannot register statically) or dynamo's own registration is NOT yet settled.
#
# NEXT DEBUG STEP: re-run with TORCHDYNAMO_VERBOSE=1 (torch offers it in the error
# itself: "Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace") to get the internal
# frame that computed the name. NOTE the launcher gotcha that wasted one boot:
# /home/dank/serve_glm_autoround.sh RESETS ENVARGS (it is initialized empty near the
# top and only the COMPILE_PATCH branch appends), so an outer `ENVARGS=... bash ...`
# is silently dropped — add the var inside the script or to the branch itself.
--- a/vllm/compilation/wrapper.py
+++ b/vllm/compilation/wrapper.py
@@ fullgraph opt-out (env VLLM_COMPILE_FULLGRAPH=0) @@