frank-default-small: use the repo chat_template.jinja so tools render in Qwen3 XML

The Avesed tokenizer_config.json ships no chat_template, so vLLM used its
built-in template, which does not render the <tool_call> instructions;
the model then emitted malformed XML (arguments baked into the function
name) and the qwen3_xml parser could not extract clean calls.
This commit is contained in:
Frank committed 2026-08-17 17:13:35 -07:00
1 parent 5108af11aa
commit 20108e5b7a
2 files changed
+9 -1

No files matched your search

+5
View File
@@ -35,6 +35,10 @@ fi
: "${NUM_SPEC_TOKENS:=5}"
: "${ENABLE_AUTO_TOOL_CHOICE:=1}"
: "${TOOL_CALL_PARSER:=qwen3_xml}"
# The Avesed tokenizer_config.json has no chat_template; vLLM would use its
# built-in template which does not render the model's XML tool format. Use
# the repo's chat_template.jinja (ships the <tool_call> instructions).
: "${CHAT_TEMPLATE:=/home/dank/models/avesed-qwen36-chat_template.jinja}"
if [[ ! -x "$VLLM_BIN" ]]; then
echo "missing vllm binary: $VLLM_BIN" >&2
@@ -59,4 +63,5 @@ exec "$VLLM_BIN" serve "$MODEL" \
--speculative-config "{\"method\":\"ngram\",\"prompt_lookup_max\":$NGRAM_LOOKUP_MAX,\"prompt_lookup_min\":$NGRAM_LOOKUP_MIN,\"num_speculative_tokens\":$NUM_SPEC_TOKENS}" \
--reasoning-parser qwen3 \
--trust-remote-code \
--chat-template "$CHAT_TEMPLATE" \
"${TOOL_ARGS[@]}"