From a818be70dc3ebed5aad1fc6f36c407964be3a575 Mon Sep 17 00:00:00 2001 From: Frank Date: Fri, 14 Aug 2026 14:26:14 -0700 Subject: [PATCH] dankclaw Qwen3.8: winner n_max=5 (13.3 t/s real), not matrix n_max=10 Realistic-prompt tuning: n_max=5 peaks ~13.3 t/s (baseline 7.2). matrix used a repetitive prompt that inflated MTP. n_max=10 is only ~8.6 t/s on real writing. --- scripts/frank-default-large.env | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/scripts/frank-default-large.env b/scripts/frank-default-large.env index 1c67057..00ed74a 100644 --- a/scripts/frank-default-large.env +++ b/scripts/frank-default-large.env @@ -12,19 +12,25 @@ PORT=7770 # ~70GB headroom, so 2x262k quantized KV fits. Verify at startup. CTX_SIZE=524288 PARALLEL=2 -# Spec: selected from the 2026-08-14 matrix (bench-spec-matrix.sh). -# WINNER: draft-mtp alone, n_max=10, p_min off -> ~30 t/s, STABLE, no -# premature stop. Layered draft-mtp,ngram-mod was faster on paper (median -# 133 t/s) but is non-deterministic (10-148 t/s) and shows a degenerate -# instant-EOS (tok=0) -> the same premature-stop class that retired DFlash. -# Rejected as production-unsafe. +# Spec: draft-mtp uses the MTP drafter BUNDLED inside the target GGUF. +# llama.cpp auto-detects it (b10411+). DRAFT_FILE stays EMPTY. +# +# Winner = draft-mtp, n_max=5, p_min off. +# Realistic-prompt tuning (2026-08-14, temp 0.8, production settings): +# n4 12.5 | n5 13.3 (PEAK) | n6 10.6 | n7 10.1 | n10 8.6 | baseline none 7.2. +# The 54-config matrix (repetitive "fleet" prompt) reported n_max=10 ~30 t/s, +# but that prompt MTP drafts almost perfectly; long drafts collapse to ~18% +# acceptance on real writing, so n_max=10 is actually ~8.6 t/s there. +# Layered draft-mtp,ngram-mod is faster on paper (matrix median 133 t/s) but +# non-deterministic (10-148 t/s) and showed a degenerate instant-EOS -> the +# same premature-stop class that retired DFlash. Rejected as production-unsafe. +# p_min off: --spec-draft-p-min 0.8 was neutral to slightly worse for draft-mtp. +# No premature-stop on realistic text at ubatch 256 (see batch notes below); +# the earlier 'premature EOS' on the repetitive bench prompt was sampling +# variance, not a corruption bug. SPEC_TYPE=draft-mtp -# Bundled MTP: Qwen3.8 keeps the MTP head in the target GGUF. llama.cpp -# auto-detects it (b10411+). DRAFT_FILE MUST stay EMPTY. DRAFT_FILE= -# Draft length + sampling floor, from the spec matrix (n_max 2..10, p_min -# off/0.8; p_min 0.8 did not help draft-mtp and slowed it slightly). -SPEC_DRAFT_N_MAX=10 +SPEC_DRAFT_N_MAX=5 SPEC_DRAFT_P_MIN= SPEC_NGRAM_MOD_N_MIN=48 SPEC_NGRAM_MOD_N_MAX=64