frank-primary: DFlash2 load attempt + fp8 verdict + compile blocker narrowed
DFlash2: fetched incoai/GLM-5.3-Flash-DFlash2 (2,342,169,800 B, sha256 matches the
Hub LFS oid) and tried --speculative-config {"method":"dflash","model":"/draft"}.
Config and drafter are accepted; the worker dies with 'Model does not support
EAGLE3 interface' - so the only gap is the target-side aux-hidden-state contract
(set_aux_hidden_state_layers + per-layer capture + the two-value forward return,
as deepseek_v2.py demonstrates twice in this image). Recorded the exact interface,
the GLM-specific unknown (what the single-stream state entering layer k means for
an mHC multi-stream trunk) and the honest expectation: no speedup on a
token-proportional pointwise bound.
fp8 KV: recorded the deciding measurement - the fork's KV reserve model is
dtype-blind (the fp8 boot printed the identical 1760 MiB / 13600 B-per-token plan),
so fp8 cannot buy capacity even if the gfx12 MLA path were rewritten. Verdict: do
not spend effort there.
Compile blocker narrowed from 'a latent GLM bug' to dynamo's name map: the
KeyError is thrown by torch/_dynamo/output_graph.py's nn_modules lookup (:3000,
registered :1259/:1384, _modules[..] prefixes stripped :1225), i.e. the compiler
cannot resolve a submodule by name while tracing the decoder layer. Next step is
TORCHDYNAMO_VERBOSE=1, with the note that the scratch launcher resets ENVARGS.
tests/run.sh: 362 seams pass.
This commit is contained in:
@@ -38,18 +38,33 @@
|
||||
#
|
||||
# NEXT BLOCKER — NOT FIXED HERE. With 01+02+03 and
|
||||
# R4D_GLM_COMPILE=1 VLLM_COMPILE_FULLGRAPH=0 VLLM_USE_AOT_COMPILE=0 the trunk traces
|
||||
# ("Dynamo bytecode transform time: 1.08 s") and then the worker dies during profile_run:
|
||||
# ("Dynamo bytecode transform time: 1.08 s") and then the worker dies during profile_run
|
||||
# on the decode forward:
|
||||
#
|
||||
# KeyError: 'in_proj_qkvbfg_a'
|
||||
# vllm/models/glm5next/common/model.py:456 in Glm5NextDecoderLayer.forward
|
||||
# model.py:748 Glm5NextModel.forward hidden_states, residual, post, comb = layer(...)
|
||||
# model.py:456 Glm5NextDecoderLayer.forward
|
||||
#
|
||||
# `in_proj_qkvbfg_a` is the KDA layer's merged column-parallel projection
|
||||
# (models/glm5next/common/kda.py:217, called at :407). The key is looked up in a
|
||||
# dict/mapping that the EAGER path never reaches, so compiling the trunk exposes a
|
||||
# latent bug in the mHC/KDA wiring rather than a dynamo limitation. Fix that and
|
||||
# whatever follows before this changelist can boot — then re-measure: the prize is the
|
||||
# 44 % pointwise mass (see 01), and the frozen-probe log line is the signal that 02 is
|
||||
# live.
|
||||
# (models/glm5next/common/kda.py:217, called at :407) — and it is also a name in the
|
||||
# load_weights shard table (model.py:780-785), i.e. a real submodule attribute name,
|
||||
# not an arbitrary string.
|
||||
#
|
||||
# WHERE IT IS THROWN (established 2026-09-30 by reading the full traceback, since the
|
||||
# report line is inside dynamo-rewritten code): torch/_dynamo keeps a name -> module map
|
||||
# and looks it up by name — output_graph.py:1259 / :1384 register into `nn_modules`,
|
||||
# :3000 does `self.nn_modules[node.target]` (a bare-name KeyError when absent), and
|
||||
# :1225 documents stripping "_buffers[..]/_parameters[..]/_modules[..]" from those
|
||||
# names. So the compiler is failing to RESOLVE A SUBMODULE BY NAME while tracing the
|
||||
# GLM decoder layer. Whether the root cause is the fork's module tree (something
|
||||
# dynamo cannot register statically) or dynamo's own registration is NOT yet settled.
|
||||
#
|
||||
# NEXT DEBUG STEP: re-run with TORCHDYNAMO_VERBOSE=1 (torch offers it in the error
|
||||
# itself: "Set TORCHDYNAMO_VERBOSE=1 for the internal stack trace") to get the internal
|
||||
# frame that computed the name. NOTE the launcher gotcha that wasted one boot:
|
||||
# /home/dank/serve_glm_autoround.sh RESETS ENVARGS (it is initialized empty near the
|
||||
# top and only the COMPILE_PATCH branch appends), so an outer `ENVARGS=... bash ...`
|
||||
# is silently dropped — add the var inside the script or to the branch itself.
|
||||
--- a/vllm/compilation/wrapper.py
|
||||
+++ b/vllm/compilation/wrapper.py
|
||||
@@ fullgraph opt-out (env VLLM_COMPILE_FULLGRAPH=0) @@
|
||||
|
||||
Reference in New Issue
Block a user