Tune frank-default-large params

- frank-default-large.env: CTX_SIZE 262144 -> 196608 (192K context),
  BATCH_SIZE/UBATCH_SIZE 4096 -> 8192, CTX_CHECKPOINTS 16 -> 32, add
  preserve_thinking:true to chat_template_kwargs
- kingjones-strix-mtp.md: sync launch excerpt + note on preserve_thinking
  and the peg-native/deepseek chat_format pairing
This commit is contained in:
Frank committed 2026-08-25 02:30:25 -07:00
1 parent cd14d7bf68
commit 7a8500f6fa
2 files changed
+17 -9

No files matched your search

+7 -1
View File
@@ -68,7 +68,9 @@ llama-server \
-m Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf \
--mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
--chat-template-file chat-template.jinja \
-ngl 999 -dev ROCm0 -c 262144 -np 1 \
--chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
-ngl 999 -dev ROCm0 -c 196608 -np 1 \
-b 8192 -ub 8192 -ctxcp 32 \
--spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf \
--spec-draft-ngl 999 --spec-draft-n-max 4 \
-fa on --no-mmap --cache-type-k f16 --cache-type-v f16
@@ -82,6 +84,10 @@ llama-server \
`enable_thinking: false` instead.
4. Small `max_tokens` with thinking on returns empty `content`. The Sharp
template drives thinking on by default (effort medium).
5. `preserve_thinking:true` (in `chat_template_kwargs`) keeps `<think>`
blocks in recorded assistant messages. `preserve_thinking:false` strips
them. See [[deployment]] for the chat_format/reasoning_format pairing
(`peg-native` + `deepseek`) the Pi agent expects from this endpoint.
## See Also