test(frank-medium): A/B DFlash2 vs MTP on FP8 — DFlash2 wins on decode

DFlash2 (100.0 t/s decode, 287.1 t/s burst-4) beats MTP self-speculation
(85.9 t/s, 179.7 t/s) despite a lower accept length (4.24 vs 5.66): the
small separate drafter steps faster than the target's own MTP layer applied
autoregressively. DFlash2 is the deployed shape; both swap-script paths now
default to dflash/7. Docs updated.
This commit is contained in:
Frank
2026-09-20 22:20:53 -07:00
parent 1ae73db084
commit f998a2d9f2
6 changed files with 111 additions and 44 deletions
@@ -110,10 +110,11 @@ Environment=MAXLEN=262144
# 400 here. Vision traffic routes to frank-small on dankclaw (harness
# modelRoles vision -> frank-small).
Environment=GPU_UTIL=0.98
# Speculative decoding: A/B test — DFlash2 external drafter at depth 7 (the
# documented DFlash2 depth), to compare against the MTP self-speculation
# baseline (depth 8, mean accept 5.66/8). The DFlash2 drafter is a separate
# quantization-agnostic checkpoint at /models/Qwen3.8-27B-DFlash2-FP8.
# Speculative decoding: DFlash2 external drafter at depth 7. A/B tested
# against MTP self-speculation (depth 8) on this FP8 checkpoint: DFlash2
# wins on decode (100.0 vs 85.9 t/s geomean, 287.1 vs 179.7 t/s burst-4)
# despite a lower accept length (4.24 vs 5.66) — the small separate drafter
# steps faster than the target's own MTP layer applied autoregressively.
Environment=SPEC_METHOD=dflash
Environment=SPEC=7
# KV cache: 0 = profile the shape (the documented plain-FP8 path's default;