dankclaw: bench cycle 1 - switch spec to ngram-mod; draft flags conditional
DFlash causes premature EOS on a generating slot when another task lands with high LCP prompt reuse (shared draft ctx corruption, reproduced 2026-08-14 3x). Benching ngram-mod vs MTP sidecar to pick the replacement. serve script now passes --spec-draft-model only for draft-* spec types.
This commit is contained in:
1 parent
42558f1707
commit
6451017d3c
2 files changed
+29
-14
No files matched your search
@@ -48,16 +48,25 @@ LLAMA_SERVER="${LLAMA_SERVER:-/home/dank/llama.cpp/build/bin/llama-server}"
|
||||
MODEL="$MODEL_DIR/$MODEL_FILE"
|
||||
DRAFT="$MODEL_DIR/$DRAFT_FILE"
|
||||
|
||||
for f in "$MODEL" "$DRAFT"; do
|
||||
if [[ ! -f "$f" ]]; then
|
||||
echo "missing required file: $f" >&2
|
||||
if [[ ! -f "$MODEL" ]]; then
|
||||
echo "missing required file: $MODEL" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Draft-model flags apply only to spec types that use a draft file
|
||||
# (draft-dflash, draft-mtp, draft-eagle3). ngram-mod is draftless.
|
||||
SPEC_ARGS=()
|
||||
if [[ "$SPEC_TYPE" == *draft-* ]]; then
|
||||
if [[ ! -f "$DRAFT" ]]; then
|
||||
echo "missing required file: $DRAFT" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
SPEC_ARGS+=(--spec-draft-model "$DRAFT")
|
||||
fi
|
||||
|
||||
exec "$LLAMA_SERVER" \
|
||||
--model "$MODEL" \
|
||||
--spec-draft-model "$DRAFT" \
|
||||
"${SPEC_ARGS[@]}" \
|
||||
--spec-type "$SPEC_TYPE" \
|
||||
--spec-draft-n-max "$SPEC_DRAFT_N_MAX" \
|
||||
--spec-ngram-mod-n-min "$SPEC_NGRAM_MOD_N_MIN" \
|
||||
|
||||
Reference in new issue
Block a user