frank-primary: DFlash2 load attempt + fp8 verdict + compile blocker narrowed
DFlash2: fetched incoai/GLM-5.3-Flash-DFlash2 (2,342,169,800 B, sha256 matches the
Hub LFS oid) and tried --speculative-config {"method":"dflash","model":"/draft"}.
Config and drafter are accepted; the worker dies with 'Model does not support
EAGLE3 interface' - so the only gap is the target-side aux-hidden-state contract
(set_aux_hidden_state_layers + per-layer capture + the two-value forward return,
as deepseek_v2.py demonstrates twice in this image). Recorded the exact interface,
the GLM-specific unknown (what the single-stream state entering layer k means for
an mHC multi-stream trunk) and the honest expectation: no speedup on a
token-proportional pointwise bound.
fp8 KV: recorded the deciding measurement - the fork's KV reserve model is
dtype-blind (the fp8 boot printed the identical 1760 MiB / 13600 B-per-token plan),
so fp8 cannot buy capacity even if the gfx12 MLA path were rewritten. Verdict: do
not spend effort there.
Compile blocker narrowed from 'a latent GLM bug' to dynamo's name map: the
KeyError is thrown by torch/_dynamo/output_graph.py's nn_modules lookup (:3000,
registered :1259/:1384, _modules[..] prefixes stripped :1225), i.e. the compiler
cannot resolve a submodule by name while tracing the decoder layer. Next step is
TORCHDYNAMO_VERBOSE=1, with the note that the scratch launcher resets ENVARGS.
tests/run.sh: 362 seams pass.
This commit is contained in:
1 parent
bd8c72ff47
commit
1c18b9c808
3 files changed
+133
-28
No files matched your search
@@ -66,6 +66,21 @@
|
||||
capture (`set_aux_hidden_state_layers` / per-layer capture, as `deepseek_v2.py`
|
||||
demonstrates). No speedup is budgeted: a verification pass pays pointwise cost
|
||||
proportional to its token count.
|
||||
- **DFlash2 tried, and the gap is exactly that one interface.** Payload fetched
|
||||
and hash-verified (`/home/dank/models/GLM-5.3-Flash-DFlash2`, 2,342,169,800 B,
|
||||
sha256 `b038e1d9…` = the Hub's LFS oid). Booting with
|
||||
`--speculative-config '{"method":"dflash","model":"/draft","num_speculative_tokens":8}'`
|
||||
accepts the config and the drafter and dies at worker init with
|
||||
`RuntimeError: Model does not support EAGLE3 interface` — nothing about the
|
||||
drafter, the method or the payload is wrong; the trunk simply does not
|
||||
implement the aux-hidden-state contract.
|
||||
- **Compile blocker narrowed to dynamo's name map.** The `KeyError:
|
||||
'in_proj_qkvbfg_a'` (see above) is thrown from `torch/_dynamo/output_graph.py`'s
|
||||
`nn_modules` name lookup (:3000, registered at :1259/:1384, with `_modules[..]`
|
||||
prefixes stripped at :1225) — i.e. the compiler cannot resolve a submodule by
|
||||
name while tracing the GLM decoder layer. Next step is `TORCHDYNAMO_VERBOSE=1`
|
||||
(torch names it in the error), noting the scratch launcher resets `ENVARGS` so
|
||||
it must be set inside the script's COMPILE_PATCH branch.
|
||||
- Measured and rejected: **MTP4** (the head loads, arch resolves to
|
||||
`Glm5NextMTPModel`, acceptance is excellent at mean length 3.76-4.90 — but
|
||||
decode was unchanged (9.6-11.2 vs 10.0) while prefill COLLAPSED 918 -> 255.
|
||||
|
||||
Reference in new issue
Block a user