frank-medium: ngram-mod is a legal chain member -- wire it and measure it
Unlike MTP, ngram-mod needs no draft context: common/speculative.cpp adds the ngram types unconditionally while the model-backed drafters are gated on ctx_dft != nullptr, so 'draft-dflash,ngram-mod' is structurally allowed (and the config vector's fixed order -- ngram first, then the drafters -- makes CLI order irrelevant). The harness grows the two things this question needs: --configs, and an ECHO shape. ngram-mod can only ever match text that is present in the prompt, so measuring it on prose calls it useless and measuring DFlash on echo calls it wasteful; both shapes now run, and the greedy text of each is compared across configurations because speculation may not change answers.
This commit is contained in:
1 parent
34253a9903
commit
21daf07713
3 files changed
+136
-56
No files matched your search
@@ -87,6 +87,9 @@ fi
|
||||
: "${SPEC_DRAFT_TYPE_V:=f16}"
|
||||
: "${SPEC_DRAFT_POLL:=1}"
|
||||
: "${SPEC_DRAFT_POLL_BATCH:=1}"
|
||||
: "${SPEC_NGRAM_N_MATCH:=24}"
|
||||
: "${SPEC_NGRAM_N_MIN:=48}"
|
||||
: "${SPEC_NGRAM_N_MAX:=64}"
|
||||
|
||||
NO_MMPROJ="${NO_MMPROJ:-1}"
|
||||
|
||||
@@ -129,6 +132,16 @@ if [[ -n "$SPEC_TYPE" && "$SPEC_TYPE" != "none" ]]; then
|
||||
if [[ "$SPEC_TYPE" == *dspark* || ( "$SPEC_DRAFT_P_MIN" != "0.0" && "$SPEC_DRAFT_P_MIN" != "0" ) ]]; then
|
||||
SPEC_ARGS+=(--spec-draft-n-min "$SPEC_DRAFT_N_MIN" --spec-draft-p-min "$SPEC_DRAFT_P_MIN")
|
||||
fi
|
||||
# ngram-mod is draftless: it needs no draft context and its window is its
|
||||
# own (--spec-ngram-mod-*), so it can ride in the chain beside a model
|
||||
# drafter. It also costs no VRAM -- a CPU hash table.
|
||||
if [[ "$SPEC_TYPE" == *ngram* ]]; then
|
||||
SPEC_ARGS+=(
|
||||
--spec-ngram-mod-n-match "$SPEC_NGRAM_N_MATCH"
|
||||
--spec-ngram-mod-n-min "$SPEC_NGRAM_N_MIN"
|
||||
--spec-ngram-mod-n-max "$SPEC_NGRAM_N_MAX"
|
||||
)
|
||||
fi
|
||||
if [[ -n "$SPEC_DRAFT_DEVICE" ]]; then
|
||||
SPEC_ARGS+=(--spec-draft-device "$SPEC_DRAFT_DEVICE")
|
||||
fi
|
||||
|
||||
Reference in new issue
Block a user