diff --git a/AGENTS.md b/AGENTS.md index d7cb73d..69e6e8a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -298,8 +298,10 @@ via the `PROMPTFORGE_*` env block (the certified runner's exact env). The serve script (`scripts/serve-frank-large.sh`) validates binary → main GGUF → 3 sidecars and exits with a pointer message if any is missing, then `exec`s `llama-server` with `--jinja --alias frank-large --chat-template-kwargs -'{"preserve_thinking":true}'` (Qwen3.8's template needs this to carry -thinking blocks across turns; a request-level value still wins). +'{"preserve_thinking":true}' --chat-template-file $TEMPLATE_FILE` (the +Qwen-Sharp v22.4.0 template; without `--chat-template-file` the binary +falls back to the built-in `peg-native` template, which produces a +~73,600-token prompt for a ~250-token request). The iGPU addresses **114 GiB** (the GTT aperture over the shared LPDDR, `gtt_total` in `/sys/class/drm/card1/device/`; `rocm-smi`'s 2 GiB "dedicated" diff --git a/serverAppConfigs-docs/LLM/frank-large-kairic-edge-qwen38-27b.md b/serverAppConfigs-docs/LLM/frank-large-kairic-edge-qwen38-27b.md index d535b6c..1a7b215 100644 --- a/serverAppConfigs-docs/LLM/frank-large-kairic-edge-qwen38-27b.md +++ b/serverAppConfigs-docs/LLM/frank-large-kairic-edge-qwen38-27b.md @@ -127,9 +127,21 @@ The three `.pfs` sidecars are wired via the `PROMPTFORGE_*` env block The serve script (`scripts/serve-frank-large.sh`) validates binary → main GGUF → 3 sidecars and exits with a pointer message if any is missing, then `exec`s `llama-server` with `--jinja --alias frank-large ---chat-template-kwargs '{"preserve_thinking":true}'` (Qwen3.8's template -needs this to carry thinking blocks across turns; a request-level value -still wins). +--chat-template-kwargs '{"preserve_thinking":true}' --chat-template-file +$TEMPLATE_FILE` (Qwen3.8's template needs `preserve_thinking` to carry +thinking blocks across turns; a request-level value still wins). + +## Chat template + +The serve script passes `--chat-template-file` pointing at +`dankserver/frank-medium/qwen-sharp-v22.4.0.jinja` (the Qwen-Sharp +v22.4.0 template, the same one frank-medium uses). Without this flag the +binary falls back to the built-in `peg-native` template, which produces a +~73,600-token prompt for a ~250-token request (280x the expected size). +With the sharp template wired in, the same request produces a 62–175-token +prompt. The `Chat format: peg-native` log line is the format-type label, +not the template indicator — the sharp template is active even when that +line appears. ## Memory budget @@ -174,6 +186,11 @@ tokens, 22.1 t/s average**. Per-question: 1.05–3.57s, 16.9–29.5 t/s. The MTP depth 4 + n-gram spec method is working (draft acceptance logged, both `ngram-mod` and `draft-mtp` active in the journal). +**Template verification** (2026-09-21): a 250-token request produces a +62–175-token prompt (not the 73,600-token prompt the built-in +`peg-native` template produces). Draft acceptance 0.79 (mean 4.14), the +model answers correctly, and reasoning lands in `reasoning_content`. + ## Failure modes worth knowing - **CK `config.h` is untracked**: a `git clean` in the CK checkout removes