dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: note 128K context and thinking on for frank-default-small

Browse Source
This commit is contained in:
Frank
2026-08-17 23:12:32 -07:00
parent 2e1a78aeb4
commit fe1d834d37
1 changed files with 5 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ciru-activefpx-promptforge.md
+5 -2
View File
@@ -131,9 +131,12 @@ Downloaded with the HF token to
### Launch profile (from the model card) ### Launch profile (from the model card)
- 64K context, one slot, f16 KV, context checkpoints (32), prompt cache - 128K context (user request 2026-08-18; model card default is 64K), one
slot, f16 KV (8 GiB at 128K, ~51.5 GiB total used of 64 GiB), context
checkpoints (32), prompt cache
- Native MTP draft, depth 4, device ROCm0 - Native MTP draft, depth 4, device ROCm0
- `--temp 0.7 --top-p 0.95`, reasoning off - `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`
puts `<think>` reasoning in `message.reasoning_content`)
- PromptForge env: `PROMPTFORGE_MODE=m2048_fused_tail1476`, - PromptForge env: `PROMPTFORGE_MODE=m2048_fused_tail1476`,
`GGML_CUDA_GRAPH_OPT=0`, `LLAMA_MTP_CPU_ARGMAX_FASTPATH=1` `GGML_CUDA_GRAPH_OPT=0`, `LLAMA_MTP_CPU_ARGMAX_FASTPATH=1`