dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

test(frank-medium): A/B DFlash2 vs MTP on FP8 — DFlash2 wins on decode

Browse Source 
DFlash2 (100.0 t/s decode, 287.1 t/s burst-4) beats MTP self-speculation
(85.9 t/s, 179.7 t/s) despite a lower accept length (4.24 vs 5.66): the
small separate drafter steps faster than the target's own MTP layer applied
autoregressively. DFlash2 is the deployed shape; both swap-script paths now
default to dflash/7. Docs updated.
This commit is contained in:
Frank
2026-09-20 22:20:53 -07:00
parent 1ae73db084
commit f998a2d9f2
6 changed files with 111 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/swap-frank-medium.sh
+2 -2
View File
@@ -91,10 +91,10 @@ quant="$(ssh "$HOST" "python3 -c 'import json,sys;print(json.load(open(sys.argv[
echo "==> Detected quantization: ${quant:-<none>}"
case "$quant" in
fp8) spec_method=mtp; default_spec=8 ;;
fp8) spec_method=dflash; default_spec=7 ;;
mxfp4|quark|"") spec_method=dflash; default_spec=7 ;;
*) spec_method=dflash; default_spec=7
echo "WARNING: unknown quant '$quant'; defaulting to the MXFP4 path" >&2 ;;
echo "WARNING: unknown quant '$quant'; defaulting to the DFlash2 path" >&2 ;;
esac
[[ -n "$spec" ]] || spec="$default_spec"