Tune frank-default-large params
- frank-default-large.env: CTX_SIZE 262144 -> 196608 (192K context), BATCH_SIZE/UBATCH_SIZE 4096 -> 8192, CTX_CHECKPOINTS 16 -> 32, add preserve_thinking:true to chat_template_kwargs - kingjones-strix-mtp.md: sync launch excerpt + note on preserve_thinking and the peg-native/deepseek chat_format pairing
This commit is contained in:
1 parent
cd14d7bf68
commit
7a8500f6fa
2 files changed
+17
-9
No files matched your search
@@ -68,7 +68,9 @@ llama-server \
|
||||
-m Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf \
|
||||
--mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
|
||||
--chat-template-file chat-template.jinja \
|
||||
-ngl 999 -dev ROCm0 -c 262144 -np 1 \
|
||||
--chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
|
||||
-ngl 999 -dev ROCm0 -c 196608 -np 1 \
|
||||
-b 8192 -ub 8192 -ctxcp 32 \
|
||||
--spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf \
|
||||
--spec-draft-ngl 999 --spec-draft-n-max 4 \
|
||||
-fa on --no-mmap --cache-type-k f16 --cache-type-v f16
|
||||
@@ -82,6 +84,10 @@ llama-server \
|
||||
`enable_thinking: false` instead.
|
||||
4. Small `max_tokens` with thinking on returns empty `content`. The Sharp
|
||||
template drives thinking on by default (effort medium).
|
||||
5. `preserve_thinking:true` (in `chat_template_kwargs`) keeps `<think>`
|
||||
blocks in recorded assistant messages. `preserve_thinking:false` strips
|
||||
them. See [[deployment]] for the chat_format/reasoning_format pairing
|
||||
(`peg-native` + `deepseek`) the Pi agent expects from this endpoint.
|
||||
|
||||
## See Also
|
||||
|
||||
|
||||
Reference in new issue
Block a user