dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

dankclaw Qwen3.8: winner n_max=5 (13.3 t/s real), not matrix n_max=10

Browse Source 
Realistic-prompt tuning: n_max=5 peaks ~13.3 t/s (baseline 7.2). matrix used a
repetitive prompt that inflated MTP. n_max=10 is only ~8.6 t/s on real writing.
This commit is contained in:
Frank
2026-08-14 14:26:14 -07:00
parent 75bb9bb53e
commit a818be70dc
1 changed files with 17 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-default-large.env
+17 -11
View File
@@ -12,19 +12,25 @@ PORT=7770
# ~70GB headroom, so 2x262k quantized KV fits. Verify at startup. # ~70GB headroom, so 2x262k quantized KV fits. Verify at startup.
CTX_SIZE=524288 CTX_SIZE=524288
PARALLEL=2 PARALLEL=2
# Spec: selected from the 2026-08-14 matrix (bench-spec-matrix.sh). # Spec: draft-mtp uses the MTP drafter BUNDLED inside the target GGUF.
# WINNER: draft-mtp alone, n_max=10, p_min off -> ~30 t/s, STABLE, no # llama.cpp auto-detects it (b10411+). DRAFT_FILE stays EMPTY.
# premature stop. Layered draft-mtp,ngram-mod was faster on paper (median #
# 133 t/s) but is non-deterministic (10-148 t/s) and shows a degenerate # Winner = draft-mtp, n_max=5, p_min off.
# instant-EOS (tok=0) -> the same premature-stop class that retired DFlash. # Realistic-prompt tuning (2026-08-14, temp 0.8, production settings):
# Rejected as production-unsafe. # n4 12.5 | n5 13.3 (PEAK) | n6 10.6 | n7 10.1 | n10 8.6 | baseline none 7.2.
# The 54-config matrix (repetitive "fleet" prompt) reported n_max=10 ~30 t/s,
# but that prompt MTP drafts almost perfectly; long drafts collapse to ~18%
# acceptance on real writing, so n_max=10 is actually ~8.6 t/s there.
# Layered draft-mtp,ngram-mod is faster on paper (matrix median 133 t/s) but
# non-deterministic (10-148 t/s) and showed a degenerate instant-EOS -> the
# same premature-stop class that retired DFlash. Rejected as production-unsafe.
# p_min off: --spec-draft-p-min 0.8 was neutral to slightly worse for draft-mtp.
# No premature-stop on realistic text at ubatch 256 (see batch notes below);
# the earlier 'premature EOS' on the repetitive bench prompt was sampling
# variance, not a corruption bug.
SPEC_TYPE=draft-mtp SPEC_TYPE=draft-mtp
# Bundled MTP: Qwen3.8 keeps the MTP head in the target GGUF. llama.cpp
# auto-detects it (b10411+). DRAFT_FILE MUST stay EMPTY.
DRAFT_FILE= DRAFT_FILE=
# Draft length + sampling floor, from the spec matrix (n_max 2..10, p_min SPEC_DRAFT_N_MAX=5
# off/0.8; p_min 0.8 did not help draft-mtp and slowed it slightly).
SPEC_DRAFT_N_MAX=10
SPEC_DRAFT_P_MIN= SPEC_DRAFT_P_MIN=
SPEC_NGRAM_MOD_N_MIN=48 SPEC_NGRAM_MOD_N_MIN=48
SPEC_NGRAM_MOD_N_MAX=64 SPEC_NGRAM_MOD_N_MAX=64