test(frank-medium): A/B DFlash2 vs MTP on FP8 — DFlash2 wins on decode
DFlash2 (100.0 t/s decode, 287.1 t/s burst-4) beats MTP self-speculation (85.9 t/s, 179.7 t/s) despite a lower accept length (4.24 vs 5.66): the small separate drafter steps faster than the target's own MTP layer applied autoregressively. DFlash2 is the deployed shape; both swap-script paths now default to dflash/7. Docs updated.
This commit is contained in:
@@ -110,10 +110,11 @@ Environment=MAXLEN=262144
|
||||
# 400 here. Vision traffic routes to frank-small on dankclaw (harness
|
||||
# modelRoles vision -> frank-small).
|
||||
Environment=GPU_UTIL=0.98
|
||||
# Speculative decoding: A/B test — DFlash2 external drafter at depth 7 (the
|
||||
# documented DFlash2 depth), to compare against the MTP self-speculation
|
||||
# baseline (depth 8, mean accept 5.66/8). The DFlash2 drafter is a separate
|
||||
# quantization-agnostic checkpoint at /models/Qwen3.8-27B-DFlash2-FP8.
|
||||
# Speculative decoding: DFlash2 external drafter at depth 7. A/B tested
|
||||
# against MTP self-speculation (depth 8) on this FP8 checkpoint: DFlash2
|
||||
# wins on decode (100.0 vs 85.9 t/s geomean, 287.1 vs 179.7 t/s burst-4)
|
||||
# despite a lower accept length (4.24 vs 5.66) — the small separate drafter
|
||||
# steps faster than the target's own MTP layer applied autoregressively.
|
||||
Environment=SPEC_METHOD=dflash
|
||||
Environment=SPEC=7
|
||||
# KV cache: 0 = profile the shape (the documented plain-FP8 path's default;
|
||||
|
||||
Reference in New Issue
Block a user