docs: reflect the 2026-08-17 evening changes (Quark model swap, tool calling, batch 8192)

docs/frank-redo-2026-08-17.md

@@ -9,7 +9,7 @@ The old scripts moved to `old/scripts/` in this repository.
 | Service | Host | Runtime | Model | Port |
 |---------|------|---------|-------|------|
 | frank-default-large | Dankserver | llama.cpp (patched FastMTP) | HauhauCS Qwen3.8-27B-Uncensored-Aggressive IQ4_XS | 7770 |
-| frank-default-small | Dankclaw | vLLM ROCm | Avesed Qwen3.6-35B-A3B INT8 W8A8 | 52625 |
+| frank-default-small | Dankclaw | vLLM ROCm | nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8 | 52625 |
 
 The RTX 3090 runs the whole 27B model standalone. No RPC worker, no
 tensor split, no NPU.
@@ -31,14 +31,20 @@ tensor split, no NPU.
   `--spec-draft-ngl all`, `--spec-draft-p-min 0`) then ngram-mod fallback
   (48/64/24).
 
-### frank-default-small — Avesed/Qwen3.6-35B-A3B-INT8-W8A8
+### frank-default-small — nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8
 
 - Base: Qwen3.6-35B-A3B (hybrid MoE, 256 experts top-8, ~3B active, 40
   layers: 10 blocks of 3 DeltaNet + 1 Attention, native 262144 ctx).
-- Quant: compressed-tensors INT8 W8A8 (symmetric per-channel weights,
-  dynamic per-token activations). Expert FFNs quantized; attention, router,
-  DeltaNet, lm_head kept bf16. Weights ~38 GB.
-- Served with vLLM ROCm, ngram speculative decoding.
+- Quant: AMD Quark W8A8 INT8 (per-channel static weights, per-token
+  dynamic activations). MoE experts stored as per-expert nn.Linear
+  triplets that match vLLM's FusedMoE mapping exactly. Weights ~35 GB;
+  the vision tower, router, lm_head stay BF16.
+- Served with vLLM ROCm, ngram speculative decoding, and tool calling.
+- Swap history: the initial deployment used
+  `Avesed/Qwen3.6-35B-A3B-INT8-W8A8` (compressed-tensors INT8). Its tool
+  calls were malformed (the model emitted `<function=get_weather(city=...)>`
+  instead of the Qwen3 XML structure). The Quark quant follows the format
+  cleanly; swapped in the evening of 2026-08-17.
 
 ## Context Math (frank-default-large, RTX 3090)
 
@@ -139,3 +145,47 @@ on gfx1151; newer vLLM releases may already include it.
 - HF token: `~/.cache/huggingface/token` on both machines (mode 600).
 - OpenWebUI (sibling repo): retarget `OPENAI_API_BASE_URL` to the
   Dankserver endpoint.
+
+## Evening Session (2026-08-17): tool calling, model swap, batch 8192
+
+Three changes landed after the initial deployment:
+
+### 1. Tool calling on frank-default-small (HTTP 400 fix)
+
+Clients (pi) send `tool_choice: "auto"`. vLLM rejects it with HTTP 400
+unless the server enables tool calling. Fixes applied:
+
+- `--enable-auto-tool-choice --tool-call-parser qwen3_xml` (the Avesed /
+  Quark chat template renders the Qwen3 XML tool format, and qwen3_xml
+  is the matching parser).
+- Custom chat template `/home/dank/models/qwen36-chat_template-tools.jinja`:
+  derived from the repo's `chat_template.jinja` (the tokenizer_config.json
+  ships no template, so vLLM would use its built-in one, which does not
+  render the XML tool instructions). The custom template adds:
+  - a concrete WORKED EXAMPLE built from the actual tool schema
+    (name + parameter keys) — models follow concrete examples much
+    better than the generic placeholder;
+  - tool requests default to non-thinking (`enable_thinking` unset + tools
+    present -> empty think block). Qwen3 tool calls only work reliably
+    with thinking off; normal requests keep thinking.
+
+Known pitfall: the earlier Avesed model still emitted malformed XML even
+with the strong template (`<function=get_weather(city=...)>`); that was
+model behavior, which motivated the swap.
+
+### 2. Model swap: Avesed INT8 W8A8 -> Quark W8A8 INT8
+
+`frank-default-small` now serves
+`nameistoken/Qwen3.6-35B-A3B-Quark-W8A8-INT8` (AMD Quark 0.11.1, ~35 GB,
+vLLM-native expert layout). The Avesed compressed-tensors quant's
+Qwen3-XML tool fidelity was poor. Verified with the Quark model: plain
+tool requests (no chat_template_kwargs) return clean
+`tool_calls: [{"name": "get_weather", "arguments": {"city": "Paris"}}]`;
+non-tool requests keep reasoning. The Avesed checkpoint stays in the HF
+cache for rollback.
+
+### 3. Batch 8192
+
+`MAX_NUM_BATCHED_TOKENS` lowered from 16384 to 8192 (user request).
+vLLM has no separate ubatch knob; the scheduler issues one batch per
+step up to this token cap.