diff --git a/AGENTS.md b/AGENTS.md index 155a2cb..b555f43 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -461,18 +461,27 @@ shape (pointwise cost scales with the token count, which is exactly why MTP4 measured flat), and neither arena size nor hit rate is a lever (the gather is 1 %). -**Fork work is staged, not deployed.** `scripts/patches/glm53-compile-0{1,2,3}*.patch` -are the compile-enablement changelist (decorate the trunk; freeze the ten libr4d -capability probes that do `os.path.exists`/`.so` lookups *inside* the traced -forward; make `fullgraph=True` opt-out). Boots established that -`functools.lru_cache` does **not** protect those probes (dynamo re-traces the -wrapped body even warm), that `suppress_errors` does **not** cover graph breaks -(only `fullgraph=False` does), and that the GLM trunk can never satisfy -`fullgraph` (its indexer metadata syncs to host at -`glm5next/amd/sparse_indexer.py:263,419`). With all three the trunk **traces** -(`Dynamo bytecode transform time: 1.08 s`) and then dies on -`KeyError: 'in_proj_qkvbfg_a'` in the mHC/KDA path — a latent bug the compile -exposes, and the remaining blocker. +**Fork work is staged, not deployed.** Two WIP changelists, both apply-verified +against the pinned image (`patch -p1` reproduces each artifact's recorded sha256): +`scripts/patches/glm53-compile-0{1..5}*.patch` (decorate the GLM trunk; freeze the ten +libr4d capability probes that stat/`.so`-probe *inside* the traced forward; make +`fullgraph=True` opt-out; take GLM off the fused-norm JIT kernel; make the indexer's +in-forward workspace acquisition compiler-opaque) and +`glm53-dflash-01-aux-hidden-states.patch` (the EAGLE3 interface, below). Boots +established that `functools.lru_cache` does **not** protect a probe (dynamo re-traces +the wrapped body even warm), that `suppress_errors` does **not** cover graph breaks +(only `fullgraph=False` does), and that the GLM trunk can never satisfy `fullgraph` +(its indexer metadata syncs to host, `glm5next/amd/sparse_indexer.py:263,419`). With +breaks genuinely tolerated the trunk **traces** (`Dynamo bytecode transform time: +1.08 s`) and the single remaining blocker is +`KeyError: 'in_proj_qkvbfg_a'` — raised in a callee dynamo inlines into +`Glm5NextDecoderLayer.forward`, **not** at any of dynamo's four `nn_modules` lookups and +**not** a literal string lookup, so it is currently a dynamo-side name-resolution +failure rather than a demonstrated fork bug (an earlier framing here said "a latent bug +the compile exposes"; that was retracted). Watch the launcher/reset gotchas: the scratch +launcher once silently dropped `VLLM_COMPILE_FULLGRAPH=0` (making four boots run +`fullgraph=True` where every break is fatal), and a `sitecustomize.py` on `PYTHONPATH` +became a fork bomb — see the runbook. **No new HIP kernels are possible** — the image ships `r4d.so` as a binary with no sources. The 9 % router is therefore **binary-blocked**: its fused path is @@ -486,10 +495,17 @@ available on this fork. and `model_type: "qwen3"` (5 dense layers, sliding attention, block 8, `target_layer_ids [5, 14, 24, 33, 42]` of 45), and the fork **already maps it** (`registry.py:636` → `qwen3_dflash2.DFlash2Qwen3ForCausalLM`; spec method -`"dflash"`). What is missing is the target side: `glm5next` implements neither +`"dflash"`). What was missing is the target side — `glm5next` implemented neither `set_aux_hidden_state_layers()` nor the per-layer aux capture that -`deepseek_v2.py` (also MLA+MoE) demonstrates. Budget no speedup: per the profile -above a verification pass pays pointwise cost proportional to its token count. +`deepseek_v2.py` (also MLA+MoE) demonstrates — and that is now written: +`scripts/patches/glm53-dflash-01-aux-hidden-states.patch` implements exactly that +contract (`EagleModelMixin` on the trunk, capture of the single-stream state entering +each requested layer, the conditional `(hidden_states, aux_hidden_states)` tuple +return, and the two delegating methods on `Glm5NextForCausalLM`). The engine's +refusal was literally `RuntimeError: Model does not support EAGLE3 interface`, so +this patch is the prerequisite for driving the drafter at all. Budget no speedup: +per the profile above, a verification pass pays pointwise cost proportional to its +token count. **MTP works here and is still rejected.** The head loads (`Resolved architecture: Glm5NextMTPModel`) and accepts well (mean accepted