dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

dankclaw: MTP wins the spec bench; raise ubatch to 4096

Browse Source 
MTP (draft-mtp, n_max 3, a4lg MTP-ONLY-Q8_0) benches 15.3-19.5 t/s vs
ngram-mod 6.7-8.7 t/s and passes the strict LCP-reuse parallel test
(sim 0.996 / f_keep 0.994 launch while another slot generates - the
pattern that killed DFlash). ubatch 4096 to use the ~33 GiB VRAM
headroom; watch peak during load.
This commit is contained in:
Frank committed 2026-08-13 22:02:40 -07:00
1 parent 5665e1ad6f
commit f203f1bc40
1 file changed
+1 -1
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-default-large.env
+1 -1
View File
@@ -39,7 +39,7 @@ CACHE_TYPE_V=q8_0
# ubatch is the compute-graph driver; larger ubatch = more VRAM, faster prefill. # ubatch is the compute-graph driver; larger ubatch = more VRAM, faster prefill.
# Re-verified 2026-08-14 during the spec swap (see benchmark doc). # Re-verified 2026-08-14 during the spec swap (see benchmark doc).
BATCH_SIZE=8192 BATCH_SIZE=8192
UBATCH_SIZE=2048 UBATCH_SIZE=4096
# Sampling settings — per HauhauCS Qwen3.6-27B model card. # Sampling settings — per HauhauCS Qwen3.6-27B model card.
# Thinking-general: temp=1.0, top_p=0.95, top_k=20, min_p=0. # Thinking-general: temp=1.0, top_p=0.95, top_k=20, min_p=0.
# Clients may override per-request; these are the server-side defaults. # Clients may override per-request; these are the server-side defaults.