docs: dflash patch v5 clears the EAGLE3 gate — wrapper is the real target

Runbook: the five-version evolution recorded, each trap named (method
hand-write, Protocol-base MRO kill, wrong-wrapper attrs). The decisive
discovery: the runner checks the LOADED instance, which for GLM-5.3-Flash
is the multimodal Glm5NextForConditionalGeneration holding the text model
as self.language_model — v4's attrs on Glm5NextForCausalLM were invisible
to it. v5 (attrs + methods on both wrappers) passes: both TP workers log
'Using Eagle3 auxiliary layers from config: (6, 15, 25, 34, ...)' and the
DFlash2 drafter resolves + loads. Log: the DFlash bullet updated from
'the trunk simply does not implement' to 'gap closed; serving end state
still unproven (window RAM contention interrupted the boot twice)'.
This commit is contained in:
Frank committed 2026-09-30 17:59:35 -07:00
1 parent b877d87588
commit a1f14a1c0f
2 files changed
+54 -27

No files matched your search

@@ -348,30 +348,49 @@ flat decode with a prefill regression like MTP's until the pointwise mass is fus
**The port exists now: `scripts/patches/glm53-dflash-01-aux-hidden-states.patch`** (WIP,
independent of the compile changelist — it patches the same file as compile-01, so
compile-01 goes first if both are applied; base `008eeaa2…` → patched `5eea89c5…`). It
compile-01 goes first if both are applied; base `008eeaa2…` → patched `bc9913a1…`). It
implements exactly the contract read out of this image: `EagleModelMixin` on
`Glm5NextModel`, capture of the single-stream `hidden_states` ENTERING each requested
layer in the loop, the conditional `(hidden_states, aux_hidden_states)` tuple return, and
`SupportsEagle3` among `Glm5NextForCausalLM`'s bases (capture shape from
`deepseek_v2.py:1551-1585`; tuple consumption at `v1/worker/gpu/cudagraph_utils.py:621`).
the three `SupportsEagle3` member attributes plus both interface methods on the
wrappers (capture shape from `deepseek_v2.py:1551-1585`; tuple consumption at
`v1/worker/gpu/cudagraph_utils.py:621`).
**The first version of that patch was wrong in an instructive way, and it is the trap to
avoid on any future spec-decode port: `supports_eagle3` is a Protocol `isinstance`
check, not duck typing.** It hand-wrote `set_aux_hidden_state_layers` and
`get_eagle3_default_aux_hidden_state_layers` on the outer wrapper — the two methods the
traceback's call site reads — and the boot reproduced the *identical*
`RuntimeError: Model does not support EAGLE3 interface`. The reason is
`interfaces.py:1663`: `SupportsEagle3` also declares
`supports_eagle3: ClassVar[Literal[True]] = True`, and a runtime-checkable Protocol
requires every member to be *present* on the instance. **Inherit the Protocol** — the
idiom every model in the image uses (`LagunaForCausalLM`, `MiMoV2FlashForCausalLM`,
`DFlashLagunaForCausalLM` all list `SupportsEagle3` in their bases) — which supplies the
flag, `SupportsEagleBase`'s `has_own_lm_head` / `has_own_embed_tokens`, *and* working
default implementations of both methods (they unwrap
`holder = getattr(parent_ref, "model", parent_ref)`, assert it is an `EagleModelMixin`,
and default to `(2, L // 2, L - 3)`). The lesson generalizes: when a runtime error names
an interface, read the interface's declaration first — the methods a call site happens to
invoke are not necessarily the whole contract.
**The patch went through five versions, and the decisive discovery was where the
instance actually lives.** The traceback's `set_eagle3_aux_hidden_state_layers(self.model)`
is called on the model the runner LOADED — and for GLM-5.3-Flash that is
`Glm5NextForConditionalGeneration` (the multimodal wrapper; the APIServer log names it),
which holds the text model as `self.language_model = Glm5NextForCausalLM`
(`glm5next/common/model.py:1113`). Patching only `Glm5NextForCausalLM` (v4) still failed
with the identical error because the `isinstance` ran against the wrapper. **v5 fixes
both wrappers** and is the first version to clear the gate (both TP workers log
`Using Eagle3 auxiliary layers from config: (6, 15, 25, 34, …)`).
The full evolution, each an instructive trap:
1. **v1** hand-wrote the two methods the traceback names → identical error. The methods
are not the whole contract.
2. **v2** inherited `SupportsEagle3` as a base (the image's own idiom — `laguna.py:717`,
`mimo_v2.py:834`, …) → dies at class creation with `Cannot create a consistent method
resolution order (MRO) for bases …`. GLM's wrappers already carry four/five
Protocol-derived bases, and simulation confirmed the ForCausalLM's five-base ordering
cannot take `SupportsEagle3` in *any* position.
3. **v3** declared the members as class attributes (MRO-safe) — but only on
`Glm5NextForCausalLM` → identical error (the wrong wrapper, see above).
4. **v4** added all three members (`supports_eagle3`, `has_own_lm_head`,
`has_own_embed_tokens` — a runtime-checkable Protocol's `isinstance` walks ALL
`__protocol_attrs__` *including base-Protocol members* via `getattr_static`, verified
experimentally in-image with stock typing) on `Glm5NextForCausalLM` → identical error.
5. **v5** adds the three attributes to BOTH wrappers + the two methods on both
(`Glm5NextForCausalLM` unwraps to `self.model` — the `EagleModelMixin` trunk;
`Glm5NextForConditionalGeneration` delegates to `self.language_model`, the
`pixtral.py:498` pattern for exactly this wrapper shape) → **gate passes**.
**The lesson generalizes twice over**: when a runtime error names an interface, read the
interface's declaration first — and establish WHICH INSTANCE the check runs against
before patching anything: multimodal wrappers (`…ForConditionalGeneration`) are loaded
models, not pass-throughs, and the text model behind `self.language_model` is invisible
to a check run on the wrapper.
**Retired diagnostic (do not resurrect as-is): the `/probe` startup hook.** An
`op_probe.py` + `sitecustomize.py` pair was mounted to wrap vLLM module forwards and