docs: document Qwen-Sharp template wiring and 73k-token diagnosis

The serve script now passes --chat-template-file pointing at the
Qwen-Sharp v22.4.0 template. Without it the binary falls back to the
built-in peg-native template, which produces a ~73,600-token prompt for
a ~250-token request. With the sharp template wired in, the same request
produces a 62-175-token prompt.
This commit is contained in:
Frank committed 2026-09-21 20:11:00 -07:00
1 parent 293e4e4b09
commit 97f60b35ef
2 files changed
+24 -5

No files matched your search

+4 -2
View File
@@ -298,8 +298,10 @@ via the `PROMPTFORGE_*` env block (the certified runner's exact env). The
serve script (`scripts/serve-frank-large.sh`) validates binary → main GGUF →
3 sidecars and exits with a pointer message if any is missing, then `exec`s
`llama-server` with `--jinja --alias frank-large --chat-template-kwargs
'{"preserve_thinking":true}'` (Qwen3.8's template needs this to carry
thinking blocks across turns; a request-level value still wins).
'{"preserve_thinking":true}' --chat-template-file $TEMPLATE_FILE` (the
Qwen-Sharp v22.4.0 template; without `--chat-template-file` the binary
falls back to the built-in `peg-native` template, which produces a
~73,600-token prompt for a ~250-token request).
The iGPU addresses **114 GiB** (the GTT aperture over the shared LPDDR,
`gtt_total` in `/sys/class/drm/card1/device/`; `rocm-smi`'s 2 GiB "dedicated"