frank-primary: compile changelist — real diffs, break diagnostics, patch 04, corrected overclaim
Working the dyno name-map blocker one step at a time, with the tools to see it:
- All four patch artifacts are now genuine diff -u patches (they were prose + pseudo
hunks before, useless to patch(1)). Verified against the pinned image: each applies
clean and reproduces its recorded patched sha256 (6e9cb81b… / 0751852e… / 732363bb…
/ 84969993…).
- New: scripts/patches/glm53-compile-04-fused-qkv-rmsnorm-optout.patch — takes GLM's
MLA block off the fused-norm JIT kernel (R4D_GLM_FUSE_QKV_RMSNORM=0; the flag is
already a constructor arg glm5next sets at attention.py:583). Reason: in
models/common/ops/fused_qk_rmsnorm.py the kernel object is bound at line 223, AFTER
its use at line 211 — the launcher family (@kernel_launcher, 39 call sites) is what
breaks dynamo's break analysis.
- New diagnostics, both reusable and recorded: TORCH_LOGS=graph_breaks +
VLLM_USE_AOT_COMPILE=0 to name each break with its user stack, and
/home/dank/dynamo-probe (sitecustomize.py on PYTHONPATH, VLLM_DYNAMO_DUMP_ATTRS=1)
to print the dynamo name-map miss detail. Gotchas: a .pth on PYTHONPATH is NOT
executed, and the scratch launcher reset ENVARGS (now ${ENVARGS:-}), which silently
dropped env vars and wasted a boot.
Measurement discipline correction: I first recorded the 'Read uninitialized cell' and
'Unsupported method call' failures as fatal blockers. They are not established as
such — they appear only when TORCH_LOGS=graph_breaks is set, while a flag-free
break-tolerance run gets past break handling and dies on the name-map KeyError. The
likely mechanism is that graph_breaks logging makes dynamo build a break
*explanation*, and that analysis itself trips on the uninitialized cell. Patch 04's
header, the runbook and the work log now carry the confound explicitly rather than
the overclaim.
tests/run.sh: 362 seams pass.
This commit is contained in:
@@ -74,13 +74,33 @@
|
||||
`RuntimeError: Model does not support EAGLE3 interface` — nothing about the
|
||||
drafter, the method or the payload is wrong; the trunk simply does not
|
||||
implement the aux-hidden-state contract.
|
||||
- **Compile blocker narrowed to dynamo's name map.** The `KeyError:
|
||||
'in_proj_qkvbfg_a'` (see above) is thrown from `torch/_dynamo/output_graph.py`'s
|
||||
`nn_modules` name lookup (:3000, registered at :1259/:1384, with `_modules[..]`
|
||||
prefixes stripped at :1225) — i.e. the compiler cannot resolve a submodule by
|
||||
name while tracing the GLM decoder layer. Next step is `TORCHDYNAMO_VERBOSE=1`
|
||||
(torch names it in the error), noting the scratch launcher resets `ENVARGS` so
|
||||
it must be set inside the script's COMPILE_PATCH branch.
|
||||
- **Compile work: one confirmed blocker, plus a measurement confound worth
|
||||
recording.** Flag-free break-tolerance runs (01+02+03, AOT off, breaks allowed)
|
||||
get past break handling and die on the dynamo name-map `KeyError:
|
||||
'in_proj_qkvbfg_a'`. Adding `TORCH_LOGS=graph_breaks` to the same config changes
|
||||
where it dies: a fatal-looking `Unsupported: Read uninitialized cell` at
|
||||
`mla.py:208` calling the fork's fused-norm JIT kernel
|
||||
(`models/common/ops/fused_qk_rmsnorm.py`: `_FUSED_Q_KV_RMSNORM_KERNEL` is bound at
|
||||
line 223, *after* its use at line 211), and with that removed, `Unsupported:
|
||||
Unsupported method call` in the indexer's in-forward workspace allocation
|
||||
(`sparse_indexer.py:132 -> workspace.py:135 get_simultaneous / :155
|
||||
_ensure_workspace_size`). **The likely mechanism is that `graph_breaks` logging
|
||||
makes dynamo build a break *explanation*, and that analysis is what trips** — so
|
||||
those two are candidates, not proven blockers, and I overclaimed them once before
|
||||
catching it. `scripts/patches/glm53-compile-04-fused-qkv-rmsnorm-optout.patch` takes
|
||||
GLM off the fused-norm kernel anyway (`R4D_GLM_FUSE_QKV_RMSNORM=0`, a constructor
|
||||
flag glm5next sets at `attention.py:583`); whether it is *needed* is what the
|
||||
flag-free 01+02+03+04 run settles.
|
||||
- **Diagnostics built for this (reusable).** `TORCH_LOGS=graph_breaks` +
|
||||
`VLLM_USE_AOT_COMPILE=0` names each break with its user stack (AOT compile
|
||||
hard-refuses breaks outright); `/home/dank/dynamo-probe` on `PYTHONPATH` with
|
||||
`VLLM_DYNAMO_DUMP_ATTRS=1` prints the dynamo name-map miss detail (node target,
|
||||
near-miss keys, whether another live `OutputGraph` holds the key). Two gotchas
|
||||
recorded: a `.pth` on `PYTHONPATH` is NOT executed (use `sitecustomize.py`,
|
||||
chaining the image's apport hook), and the scratch launcher used to reset
|
||||
`ENVARGS` (now `${ENVARGS:-}`), which silently dropped env vars and wasted a boot.
|
||||
All four patch artifacts are now genuine `diff -u` patches, verified to apply to
|
||||
the pinned image and to reproduce their recorded sha256.
|
||||
- Measured and rejected: **MTP4** (the head loads, arch resolves to
|
||||
`Glm5NextMTPModel`, acceptance is excellent at mean length 3.76-4.90 — but
|
||||
decode was unchanged (9.6-11.2 vs 10.0) while prefill COLLAPSED 918 -> 255.
|
||||
|
||||
Reference in New Issue
Block a user