test(frank-medium): A/B DFlash2 vs MTP on FP8 — DFlash2 wins on decode
DFlash2 (100.0 t/s decode, 287.1 t/s burst-4) beats MTP self-speculation (85.9 t/s, 179.7 t/s) despite a lower accept length (4.24 vs 5.66): the small separate drafter steps faster than the target's own MTP layer applied autoregressively. DFlash2 is the deployed shape; both swap-script paths now default to dflash/7. Docs updated.
This commit is contained in:
1 parent
1ae73db084
commit
f998a2d9f2
6 files changed
+111
-44
No files matched your search
@@ -11,6 +11,33 @@ timestamp: 2026-09-14
|
||||
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-09-20 — DFlash2 vs MTP A/B on the FP8 checkpoint: DFlash2 wins on decode
|
||||
|
||||
- **The test.** The FP8 checkpoint carries its own single-layer MTP head, so
|
||||
it can self-speculate (`SPEC_METHOD=mtp`) or use the z-lab DFlash2 external
|
||||
drafter (`SPEC_METHOD=dflash`). Both were deployed and measured on the same
|
||||
FP8 target (`scripts/bench-conversation.py`, 10 real prompts, `max_tokens=512`,
|
||||
greedy, salted, `--burst 4`):
|
||||
- **DFlash2** (depth 7): mean accept 4.24/7, geomean decode **100.0 t/s**,
|
||||
TTFT 0.19 s, prefill 923.0 t/s, burst-4 **287.1 t/s**, KV pool 384,238 (1.47×).
|
||||
- **MTP** (depth 8): mean accept 5.66/8, geomean decode 85.9 t/s, TTFT 0.06 s,
|
||||
prefill 1096.9 t/s, burst-4 179.7 t/s, KV pool 461,540 (1.76×).
|
||||
- **Verdict: DFlash2 wins on decode (+16 %) and burst-4 (+60 %); MTP wins on
|
||||
TTFT and prefill.** The counterintuitive part: DFlash2 accepts *fewer*
|
||||
tokens per step (4.24 vs 5.66) yet decodes *faster*. Accept length is a
|
||||
proxy, not the metric — throughput is `(accept + 1) / (draft time + verify
|
||||
time)`. The DFlash2 drafter is a small separate model that steps fast; the
|
||||
MTP head is the target's own layer applied autoregressively 8×, so its draft
|
||||
is slower. The faster step more than makes up the lower accept length.
|
||||
- **FP8 + DFlash2 (100.0 t/s) beats the original MXFP4 + DFlash2 (92.4 t/s)**
|
||||
— the higher-precision FP8 target verifies more of the drafter's proposals.
|
||||
The drafter costs 2.1 GiB of VRAM (KV pool 1.47× vs MTP's 1.76×), an
|
||||
acceptable trade for the decode win.
|
||||
- **DFlash2 is the deployed shape.** The quadlet comment, the radiance
|
||||
runbook (deployed-flags table, KV pool, a new A/B section), AGENTS.md
|
||||
(topology row + frank-medium section), and the model-swap runbook + swap
|
||||
script (both paths now default to `dflash/7`) all updated.
|
||||
|
||||
## 2026-09-20 — frank-medium checkpoint swapped MXFP4 → FP8; the swap is now one command
|
||||
|
||||
- **The swap.** `just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq` (MXFP4,
|
||||
|
||||
Reference in new issue
Block a user