dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

dankclaw: FINAL spec - draft-mtp n_max 3, ubatch 8192

Browse Source 
n_max 2 benched 12.4-14.3 t/s (slower than n_max 3 at 15.3-19.5);
n_max 3 is the MTP sweet spot. ubatch 8192 (==batch) uses the VRAM
headroom (65.5 GiB of 96 GiB) safely. MTP beats ngram-mod (~7 t/s) and
matches DFlash (~20 t/s) without the parallel premature-EOS bug.
This commit is contained in:
Frank
2026-08-13 23:07:38 -07:00
parent 01eb957f90
commit a27839519e
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-default-large.env
+1 -1
View File
@@ -19,7 +19,7 @@ SPEC_TYPE=draft-mtp
# DFlash: Qwen3.6-27B-DFlash-Q8_0.gguf ; MTP: Qwen3.6-27B-MTP-ONLY-Q8_0.gguf # DFlash: Qwen3.6-27B-DFlash-Q8_0.gguf ; MTP: Qwen3.6-27B-MTP-ONLY-Q8_0.gguf
DRAFT_FILE=Qwen3.6-27B-MTP-ONLY-Q8_0.gguf DRAFT_FILE=Qwen3.6-27B-MTP-ONLY-Q8_0.gguf
# Draft length per spec type: dflash 8, mtp 2-3. # Draft length per spec type: dflash 8, mtp 2-3.
SPEC_DRAFT_N_MAX=2 SPEC_DRAFT_N_MAX=3
SPEC_NGRAM_MOD_N_MIN=48 SPEC_NGRAM_MOD_N_MIN=48
SPEC_NGRAM_MOD_N_MAX=64 SPEC_NGRAM_MOD_N_MAX=64
SPEC_NGRAM_MOD_N_MATCH=24 SPEC_NGRAM_MOD_N_MATCH=24