frank-large: wire Qwen-Sharp v22.4.0 template via --chat-template-file

The serve script only passed --chat-template-kwargs, so the binary fell
back to the built-in peg-native template. A 250-token request produced a
73,600-token prompt (280x the expected ~260). Wiring in --chat-template-file
drops the prompt to 62 tokens. Draft acceptance 0.79 (mean 4.14), model
answers correctly, reasoning lands in reasoning_content.
This commit is contained in:
Frank
2026-09-21 20:07:13 -07:00
parent b3a3b936bd
commit 293e4e4b09
+2 -1
View File
@@ -50,7 +50,7 @@ fi
: "${PARALLEL_SLOTS:=1}"
: "${SPEC_DRAFT_N_MAX:=4}"
: "${COMPATIBILITY_MODE:=1}"
: "${CACHE_RAM:=8192}"
TEMPLATE_FILE="${TEMPLATE_FILE:-$SCRIPT_DIR/../dankserver/frank-medium/qwen-sharp-v22.4.0.jinja}"
SERVER_BIN="$RUNTIME_DIR/bin/llama-server"
MODEL_FILE="$MODEL_DIR/Qwen3.8-27B-IU4-Kairic-Edge.gguf"
@@ -113,6 +113,7 @@ ARGS=(
-m "$MODEL_FILE" --alias frank-large
--host "$HOST" --port "$PORT" --jinja
--chat-template-kwargs '{"preserve_thinking":true}'
--chat-template-file "$TEMPLATE_FILE"
-dev ROCm0 -ngl 999
-c "$CONTEXT_SIZE" -b 2048 -ub 512 -fa on -ctk f16 -ctv f16
-t 16 -tb 32 -np "$PARALLEL_SLOTS" -ctxcp 32 --cache-ram "$CACHE_RAM"