Commit Graph
4 Commits
Author SHA1 Message Date
Frank 20108e5b7a frank-default-small: use the repo chat_template.jinja so tools render in Qwen3 XML
The Avesed tokenizer_config.json ships no chat_template, so vLLM used its
built-in template, which does not render the <tool_call> instructions;
the model then emitted malformed XML (arguments baked into the function
name) and the qwen3_xml parser could not extract clean calls.
2026-08-17 17:13:35 -07:00
Frank 5108af11aa frank-default-small: enable tool calling (--enable-auto-tool-choice --tool-call-parser qwen3_xml)
Clients send tool_choice=auto; without these flags vLLM rejects with
HTTP 400. The Avesed chat template renders the Qwen3 XML tool format,
so qwen3_xml is the matching parser.
2026-08-17 16:50:23 -07:00
Frank 27788f6f2c fix: make serve scripts + deploy.sh executable 2026-08-17 15:24:40 -07:00
Frank d9af7f1d19 fleet redo 2026-08-17: retire PP+NPU, new topology
- Move all previous scripts to old/scripts/ (retired PP/NPU-era)
- frank-default-large: llama.cpp FastMTP on dankserver (HauhauCS
  Qwen3.8-27B IQ4_XS + FastMTP 32K draft + ngram-mod, K q8_0/V f16,
  128K ctx, batch 2048/ubatch 1024, port 7770)
- frank-default-small: vLLM ROCm on dankclaw (Avesed Qwen3.6-35B-A3B
  INT8 W8A8, full 262144 ctx, no KV quant, 4 seqs, batch 16384,
  ngram spec, port 52625)
- retarget deploy.sh (dankserver/dankclaw, inline sudo, no helpers)
- docs: deployment + index + AGENTS.md + README for new topology,
  superseded banners on retired docs, add frank-redo-2026-08-17 runbook
2026-08-17 15:23:46 -07:00