patches: glm53-dflash-01 v2 -- satisfy SupportsEagle3 by inheriting the Protocol
v1 hand-wrote set_aux_hidden_state_layers/get_eagle3_default_aux_hidden_state_layers and the boot reproduced the identical 'Model does not support EAGLE3 interface'. Reason: supports_eagle3() is an isinstance check against a runtime-checkable Protocol whose members include a required ClassVar flag (interfaces.py:1663 supports_eagle3: ClassVar[Literal[True]] = True), so methods alone do not satisfy it. Inheriting SupportsEagle3 -- the idiom every model in this image uses, including the fork's own DFlashLagunaForCausalLM -- supplies the flag, SupportsEagleBase's has_own_lm_head/has_own_embed_tokens, and working default implementations of both methods; the hand-written methods are deleted. base 008eeaa2... -> patched 5eea89c5..., apply-verified against the pinned image. Runbook: corrected method name, recorded the Protocol trap as the generalizable lesson for future spec-decode ports.
This commit is contained in:
@@ -299,7 +299,7 @@ The fork already maps that architecture:
|
||||
and the spec method is `"dflash"` (`config/speculative.py:67`). So the port is
|
||||
**target-side only**: the GLM trunk must expose auxiliary hidden states at those five
|
||||
layer indices, the same interface `deepseek_v2.py` (also MLA+MoE) implements —
|
||||
`set_aux_hidden_state_layers()` / `get_eagle3_aux_hidden_state_layers()` plus
|
||||
`set_aux_hidden_state_layers()` / `get_eagle3_default_aux_hidden_state_layers()` plus
|
||||
capture at `idx in self.aux_hidden_state_layers` in the layer loop. `glm5next`
|
||||
implements neither.
|
||||
|
||||
@@ -316,7 +316,7 @@ RuntimeError: Model does not support EAGLE3 interface
|
||||
So nothing about the drafter, the method name, or the payload is wrong — the GLM
|
||||
trunk simply does not implement the aux-hidden-state contract the dflash runner
|
||||
requires. The reference is in this same image, twice: `deepseek_v2.py`
|
||||
(`set_aux_hidden_state_layers` / `get_eagle3_aux_hidden_state_layers` at :1948-1951,
|
||||
(`set_aux_hidden_state_layers` / `get_eagle3_default_aux_hidden_state_layers` at :1948-1951,
|
||||
capture at :1554 and :1579) and the fork's own `qwen4_exp` trunk. The contract is
|
||||
small:
|
||||
|
||||
@@ -327,7 +327,7 @@ small:
|
||||
rank holds fewer tokens than `positions`);
|
||||
- `Glm5NextModel.forward` returns `(hidden_states, aux_hidden_states)` when the
|
||||
list is non-empty;
|
||||
- `get_eagle3_aux_hidden_state_layers()` supplies a default triple when the drafter
|
||||
- `get_eagle3_default_aux_hidden_state_layers()` supplies a default triple when the drafter
|
||||
config omits `target_layer_ids` (this drafter provides `[5, 14, 24, 33, 42]`, so
|
||||
the runner derives trunk indices `[6, 15, 25, 34, 43]`).
|
||||
|
||||
@@ -348,14 +348,30 @@ flat decode with a prefill regression like MTP's until the pointwise mass is fus
|
||||
|
||||
**The port exists now: `scripts/patches/glm53-dflash-01-aux-hidden-states.patch`** (WIP,
|
||||
independent of the compile changelist — it patches the same file as compile-01, so
|
||||
compile-01 goes first if both are applied). It implements exactly the contract read out
|
||||
of this image: `EagleModelMixin` on `Glm5NextModel`, capture of the single-stream
|
||||
`hidden_states` ENTERING each requested layer in the loop, the conditional
|
||||
`(hidden_states, aux_hidden_states)` tuple return, and
|
||||
`set_aux_hidden_state_layers` / `get_eagle3_default_aux_hidden_state_layers` on
|
||||
`Glm5NextForCausalLM` (delegation shape from `kimi_k25.py:466`, `pixtral.py:503`;
|
||||
capture shape from `deepseek_v2.py:1551-1585`; tuple consumption at
|
||||
`v1/worker/gpu/cudagraph_utils.py:621`).
|
||||
compile-01 goes first if both are applied; base `008eeaa2…` → patched `5eea89c5…`). It
|
||||
implements exactly the contract read out of this image: `EagleModelMixin` on
|
||||
`Glm5NextModel`, capture of the single-stream `hidden_states` ENTERING each requested
|
||||
layer in the loop, the conditional `(hidden_states, aux_hidden_states)` tuple return, and
|
||||
`SupportsEagle3` among `Glm5NextForCausalLM`'s bases (capture shape from
|
||||
`deepseek_v2.py:1551-1585`; tuple consumption at `v1/worker/gpu/cudagraph_utils.py:621`).
|
||||
|
||||
**The first version of that patch was wrong in an instructive way, and it is the trap to
|
||||
avoid on any future spec-decode port: `supports_eagle3` is a Protocol `isinstance`
|
||||
check, not duck typing.** It hand-wrote `set_aux_hidden_state_layers` and
|
||||
`get_eagle3_default_aux_hidden_state_layers` on the outer wrapper — the two methods the
|
||||
traceback's call site reads — and the boot reproduced the *identical*
|
||||
`RuntimeError: Model does not support EAGLE3 interface`. The reason is
|
||||
`interfaces.py:1663`: `SupportsEagle3` also declares
|
||||
`supports_eagle3: ClassVar[Literal[True]] = True`, and a runtime-checkable Protocol
|
||||
requires every member to be *present* on the instance. **Inherit the Protocol** — the
|
||||
idiom every model in the image uses (`LagunaForCausalLM`, `MiMoV2FlashForCausalLM`,
|
||||
`DFlashLagunaForCausalLM` all list `SupportsEagle3` in their bases) — which supplies the
|
||||
flag, `SupportsEagleBase`'s `has_own_lm_head` / `has_own_embed_tokens`, *and* working
|
||||
default implementations of both methods (they unwrap
|
||||
`holder = getattr(parent_ref, "model", parent_ref)`, assert it is an `EagleModelMixin`,
|
||||
and default to `(2, L // 2, L - 3)`). The lesson generalizes: when a runtime error names
|
||||
an interface, read the interface's declaration first — the methods a call site happens to
|
||||
invoke are not necessarily the whole contract.
|
||||
|
||||
**Retired diagnostic (do not resurrect as-is): the `/probe` startup hook.** An
|
||||
`op_probe.py` + `sitecustomize.py` pair was mounted to wrap vLLM module forwards and
|
||||
|
||||
Reference in New Issue
Block a user