frank-medium: ngram-mod is a legal chain member -- wire it and measure it
Unlike MTP, ngram-mod needs no draft context: common/speculative.cpp adds the ngram types unconditionally while the model-backed drafters are gated on ctx_dft != nullptr, so 'draft-dflash,ngram-mod' is structurally allowed (and the config vector's fixed order -- ngram first, then the drafters -- makes CLI order irrelevant). The harness grows the two things this question needs: --configs, and an ECHO shape. ngram-mod can only ever match text that is present in the prompt, so measuring it on prose calls it useless and measuring DFlash on echo calls it wasteful; both shapes now run, and the greedy text of each is compared across configurations because speculation may not change answers.
This commit is contained in:
1 parent
34253a9903
commit
21daf07713
3 files changed
+136
-56
No files matched your search
@@ -175,6 +175,16 @@ SPEC_DRAFT_TYPE_V=f16
|
||||
# the target model ..."), so a chain of the two is a structural dead end, not a
|
||||
# tuning question. See serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md.
|
||||
|
||||
# ngram-mod: the draftless n-gram matcher (CPU hash table, no VRAM, no model
|
||||
# file). It is a legitimate second chain member -- unlike MTP it needs no draft
|
||||
# context, so `--spec-type draft-dflash,ngram-mod` is structurally allowed. The
|
||||
# window below is the fleet's tuned one (n_min/n_max 48/64 match long echoes,
|
||||
# which is the only traffic where n-grams pay). Measured against DFlash2 on
|
||||
# 2026-09-16 -- see the runbook for the verdict before enabling it.
|
||||
SPEC_NGRAM_N_MATCH=24
|
||||
SPEC_NGRAM_N_MIN=48
|
||||
SPEC_NGRAM_N_MAX=64
|
||||
|
||||
# Text only: the server must not autoload a vision tower. Vision stays on
|
||||
# frank-small (Dankclaw, RTX 3090), which is the fleet's image endpoint.
|
||||
NO_MMPROJ=1
|
||||
Reference in new issue
Block a user