dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

test(frank-medium): A/B — flip to DFlash2 drafter (SPEC_METHOD=dflash, SPEC=7)

Browse Source
This commit is contained in:
Frank
2026-09-20 22:06:26 -07:00
parent c1a4ccbbcd
commit 1ae73db084
1 changed files with 6 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dankserver/frank-medium/quadlet/frank-medium.container
+6 -5
View File
@@ -110,11 +110,12 @@ Environment=MAXLEN=262144
# 400 here. Vision traffic routes to frank-small on dankclaw (harness # 400 here. Vision traffic routes to frank-small on dankclaw (harness
# modelRoles vision -> frank-small). # modelRoles vision -> frank-small).
Environment=GPU_UTIL=0.98 Environment=GPU_UTIL=0.98
# Speculative decoding: MTP self-speculation at depth 8 (the model's own # Speculative decoding: A/B test — DFlash2 external drafter at depth 7 (the
# single-layer MTP head, applied autoregressively). The documented plain-FP8 # documented DFlash2 depth), to compare against the MTP self-speculation
# path (upstream docker-compose) uses num_speculative_tokens=8. # baseline (depth 8, mean accept 5.66/8). The DFlash2 drafter is a separate
Environment=SPEC_METHOD=mtp # quantization-agnostic checkpoint at /models/Qwen3.8-27B-DFlash2-FP8.
Environment=SPEC=8 Environment=SPEC_METHOD=dflash
Environment=SPEC=7
# KV cache: 0 = profile the shape (the documented plain-FP8 path's default; # KV cache: 0 = profile the shape (the documented plain-FP8 path's default;
# the upstream docker-compose leaves the pin commented out). The measured pin # the upstream docker-compose leaves the pin commented out). The measured pin
# for this shape is 15989735424 (14.89 GiB, upstream, 2x R9700 32GB); after # for this shape is 15989735424 (14.89 GiB, upstream, 2x R9700 32GB); after