dankclaw: production spec -> 3-layer chain draft-mtp,ngram-mod,ngram-map-k4v (user request; ~equal real-prose speed)
This commit is contained in:
@@ -12,23 +12,15 @@ PORT=7770
|
||||
# ~70GB headroom, so 2x262k quantized KV fits. Verify at startup.
|
||||
CTX_SIZE=524288
|
||||
PARALLEL=2
|
||||
# Spec: draft-mtp uses the MTP drafter BUNDLED inside the target GGUF.
|
||||
# llama.cpp auto-detects it (b10411+). DRAFT_FILE stays EMPTY.
|
||||
#
|
||||
# Winner = draft-mtp, n_max=5, p_min off.
|
||||
# Realistic-prompt tuning (2026-08-14, temp 0.8, production settings):
|
||||
# n4 12.5 | n5 13.3 (PEAK) | n6 10.6 | n7 10.1 | n10 8.6 | baseline none 7.2.
|
||||
# The 54-config matrix (repetitive "fleet" prompt) reported n_max=10 ~30 t/s,
|
||||
# but that prompt MTP drafts almost perfectly; long drafts collapse to ~18%
|
||||
# acceptance on real writing, so n_max=10 is actually ~8.6 t/s there.
|
||||
# Layered draft-mtp,ngram-mod is faster on paper (matrix median 133 t/s) but
|
||||
# non-deterministic (10-148 t/s) and showed a degenerate instant-EOS -> the
|
||||
# same premature-stop class that retired DFlash. Rejected as production-unsafe.
|
||||
# p_min off: --spec-draft-p-min 0.8 was neutral to slightly worse for draft-mtp.
|
||||
# No premature-stop on realistic text at ubatch 256 (see batch notes below);
|
||||
# the earlier 'premature EOS' on the repetitive bench prompt was sampling
|
||||
# variance, not a corruption bug.
|
||||
SPEC_TYPE=draft-mtp
|
||||
# Spec: 3-layer chain, chosen by the user 2026-08-14 (aesthetic/short-term
|
||||
# experiment): draft-mtp first, then ngram-mod, then ngram-map-k4v as
|
||||
# fallbacks. On real prose this benches ~equal to draft-mtp alone
|
||||
# (10.4 vs ~10 t/s @16k) - the ngram layers only pay off on self-similar text.
|
||||
# Bundled MTP drafter; llama.cpp auto-detects it (b10411+). DRAFT_FILE empty.
|
||||
# n_max=5 is the realistic-text peak (n4 12.5, n6 10.6, n10 8.6).
|
||||
# Occasional slow rounds possible when fallbacks misfire; revert to
|
||||
# draft-mtp (alone) if the assistant reports stalls.
|
||||
SPEC_TYPE=draft-mtp,ngram-mod,ngram-map-k4v
|
||||
DRAFT_FILE=
|
||||
SPEC_DRAFT_N_MAX=5
|
||||
SPEC_DRAFT_P_MIN=
|
||||
|
||||
Reference in New Issue
Block a user