diff --git a/AGENTS.md b/AGENTS.md index 749f835..923b4f4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -348,7 +348,9 @@ full window; the DFlash2 drafter's 2.1 GiB costs the pool vs MTP's 461,540). DFlash2 was A/B tested against MTP self-speculation on this checkpoint and wins on decode (100.0 vs 85.9 t/s) — see the runbook. Tools via `qwen3_coder`, reasoning via `qwen3` (the field is `reasoning`, not llama.cpp's -`reasoning_content`), vision through the Gemma4 merge path. +`reasoning_content`), vision through the Gemma4 merge path, chat template +Qwen-Sharp v22.4.0 (froggeric, repo-tracked at `dankserver/frank-medium/`, +terse system prompt by default). **The entrypoint is quantization-aware.** `radiance-entrypoint.sh` reads `quantization_config.quant_method` from the checkpoint's `config.json` and diff --git a/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md b/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md index 25abbaf..8fc64e8 100644 --- a/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md +++ b/serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md @@ -47,7 +47,7 @@ board. The collective that actually runs is vLLM's R4D one-shot allreduce, which does not use RCCL at all (TheRock RCCL has no gfx1201 device code; see [[Dual-R9700-Cooperation]]). -## Deployed flags (2026-09-20) +## Deployed flags (2026-09-21) From `dankserver/frank-medium/quadlet/frank-medium.container`: @@ -64,7 +64,7 @@ From `dankserver/frank-medium/quadlet/frank-medium.container`: | Parsers | `--enable-auto-tool-choice --tool-call-parser qwen3_coder`, `--reasoning-parser qwen3` | | Sampler floor | `--override-sampling-effects` (temp 0.7 / top_p 0.95 / top_k 20) | | Multimodal | `--limit-mm-per-prompt '{"image": 8}'` | -| Chat template | `qwen-fixed-v22.3.jinja` (patch dir bind-mounted, `MM_MERGE_ENGINE=AXROW_SEPARATE` for Gemma4 vision) | +| Chat template | `qwen-sharp-v22.4.0.jinja` (froggeric Qwen-Sharp v22.4.0, repo-tracked at `dankserver/frank-medium/`, mounted at `/templates/`; terse system prompt by default — opt out per-request with `chat_template_kwargs {"terse": false}`; rollback: flip `CHAT_TEMPLATE` back to `/patches/qwen-fixed-v22.3.jinja`) | **The KV pool is auto-sized, not pinned.** Production ships no `KV_MEM` pin (`KV_MEM=0`): the pool comes up at whatever VRAM the weights + drafter leave.