patches: glm53-dflash-01 v2 -- satisfy SupportsEagle3 by inheriting the Protocol

v1 hand-wrote set_aux_hidden_state_layers/get_eagle3_default_aux_hidden_state_layers
and the boot reproduced the identical 'Model does not support EAGLE3 interface'.
Reason: supports_eagle3() is an isinstance check against a runtime-checkable
Protocol whose members include a required ClassVar flag
(interfaces.py:1663 supports_eagle3: ClassVar[Literal[True]] = True), so methods
alone do not satisfy it. Inheriting SupportsEagle3 -- the idiom every model in
this image uses, including the fork's own DFlashLagunaForCausalLM -- supplies the
flag, SupportsEagleBase's has_own_lm_head/has_own_embed_tokens, and working
default implementations of both methods; the hand-written methods are deleted.

base 008eeaa2... -> patched 5eea89c5..., apply-verified against the pinned image.
Runbook: corrected method name, recorded the Protocol trap as the generalizable
lesson for future spec-decode ports.
This commit is contained in:
Frank
2026-09-30 16:53:24 -07:00
parent ce280e7295
commit 9c5c2494da
2 changed files with 63 additions and 45 deletions
@@ -299,7 +299,7 @@ The fork already maps that architecture:
and the spec method is `"dflash"` (`config/speculative.py:67`). So the port is
**target-side only**: the GLM trunk must expose auxiliary hidden states at those five
layer indices, the same interface `deepseek_v2.py` (also MLA+MoE) implements —
`set_aux_hidden_state_layers()` / `get_eagle3_aux_hidden_state_layers()` plus
`set_aux_hidden_state_layers()` / `get_eagle3_default_aux_hidden_state_layers()` plus
capture at `idx in self.aux_hidden_state_layers` in the layer loop. `glm5next`
implements neither.
@@ -316,7 +316,7 @@ RuntimeError: Model does not support EAGLE3 interface
So nothing about the drafter, the method name, or the payload is wrong — the GLM
trunk simply does not implement the aux-hidden-state contract the dflash runner
requires. The reference is in this same image, twice: `deepseek_v2.py`
(`set_aux_hidden_state_layers` / `get_eagle3_aux_hidden_state_layers` at :1948-1951,
(`set_aux_hidden_state_layers` / `get_eagle3_default_aux_hidden_state_layers` at :1948-1951,
capture at :1554 and :1579) and the fork's own `qwen4_exp` trunk. The contract is
small:
@@ -327,7 +327,7 @@ small:
rank holds fewer tokens than `positions`);
- `Glm5NextModel.forward` returns `(hidden_states, aux_hidden_states)` when the
list is non-empty;
- `get_eagle3_aux_hidden_state_layers()` supplies a default triple when the drafter
- `get_eagle3_default_aux_hidden_state_layers()` supplies a default triple when the drafter
config omits `target_layer_ids` (this drafter provides `[5, 14, 24, 33, 42]`, so
the runner derives trunk indices `[6, 15, 25, 34, 43]`).
@@ -348,14 +348,30 @@ flat decode with a prefill regression like MTP's until the pointwise mass is fus
**The port exists now: `scripts/patches/glm53-dflash-01-aux-hidden-states.patch`** (WIP,
independent of the compile changelist — it patches the same file as compile-01, so
compile-01 goes first if both are applied). It implements exactly the contract read out
of this image: `EagleModelMixin` on `Glm5NextModel`, capture of the single-stream
`hidden_states` ENTERING each requested layer in the loop, the conditional
`(hidden_states, aux_hidden_states)` tuple return, and
`set_aux_hidden_state_layers` / `get_eagle3_default_aux_hidden_state_layers` on
`Glm5NextForCausalLM` (delegation shape from `kimi_k25.py:466`, `pixtral.py:503`;
capture shape from `deepseek_v2.py:1551-1585`; tuple consumption at
`v1/worker/gpu/cudagraph_utils.py:621`).
compile-01 goes first if both are applied; base `008eeaa2…` → patched `5eea89c5…`). It
implements exactly the contract read out of this image: `EagleModelMixin` on
`Glm5NextModel`, capture of the single-stream `hidden_states` ENTERING each requested
layer in the loop, the conditional `(hidden_states, aux_hidden_states)` tuple return, and
`SupportsEagle3` among `Glm5NextForCausalLM`'s bases (capture shape from
`deepseek_v2.py:1551-1585`; tuple consumption at `v1/worker/gpu/cudagraph_utils.py:621`).
**The first version of that patch was wrong in an instructive way, and it is the trap to
avoid on any future spec-decode port: `supports_eagle3` is a Protocol `isinstance`
check, not duck typing.** It hand-wrote `set_aux_hidden_state_layers` and
`get_eagle3_default_aux_hidden_state_layers` on the outer wrapper — the two methods the
traceback's call site reads — and the boot reproduced the *identical*
`RuntimeError: Model does not support EAGLE3 interface`. The reason is
`interfaces.py:1663`: `SupportsEagle3` also declares
`supports_eagle3: ClassVar[Literal[True]] = True`, and a runtime-checkable Protocol
requires every member to be *present* on the instance. **Inherit the Protocol** — the
idiom every model in the image uses (`LagunaForCausalLM`, `MiMoV2FlashForCausalLM`,
`DFlashLagunaForCausalLM` all list `SupportsEagle3` in their bases) — which supplies the
flag, `SupportsEagleBase`'s `has_own_lm_head` / `has_own_embed_tokens`, *and* working
default implementations of both methods (they unwrap
`holder = getattr(parent_ref, "model", parent_ref)`, assert it is an `EagleModelMixin`,
and default to `(2, L // 2, L - 3)`). The lesson generalizes: when a runtime error names
an interface, read the interface's declaration first — the methods a call site happens to
invoke are not necessarily the whole contract.
**Retired diagnostic (do not resurrect as-is): the `/probe` startup hook.** An
`op_probe.py` + `sitecustomize.py` pair was mounted to wrap vLLM module forwards and