frank-default-small: use the repo chat_template.jinja so tools render in Qwen3 XML

The Avesed tokenizer_config.json ships no chat_template, so vLLM used its
built-in template, which does not render the <tool_call> instructions;
the model then emitted malformed XML (arguments baked into the function
name) and the qwen3_xml parser could not extract clean calls.
This commit is contained in:
Frank committed 2026-08-17 17:13:35 -07:00
1 parent 5108af11aa
commit 20108e5b7a
2 files changed
+9 -1

No files matched your search

+4 -1
View File
@@ -42,9 +42,12 @@ NUM_SPEC_TOKENS=5
# Tool calling. Clients (pi, OpenWebUI) send tool_choice=auto; without
# --enable-auto-tool-choice vLLM rejects it with HTTP 400. The Avesed
# chat template renders tools in the Qwen3 XML format, so the qwen3_xml
# parser is the matching one.
# parser is the matching one. CHAT_TEMPLATE must point at the repo's
# chat_template.jinja (the tokenizer_config.json ships none; vLLM's
# built-in template does not render the XML tool instructions).
ENABLE_AUTO_TOOL_CHOICE=1
TOOL_CALL_PARSER=qwen3_xml
CHAT_TEMPLATE=/home/dank/models/avesed-qwen36-chat_template.jinja
# RDNA flash-attention for the vision encoder (and general FA paths).
# Without this, the mm encoder falls back to torch SDPA and the warmup