dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-default-large: full native 262144 context per slot (524288 total)

Browse Source 
llama.cpp splits -c across -np slots; to give each of the 2 slots the full
262144 native context, set CTX_SIZE=524288. KV ~32 GiB; fits the 96 GiB
unified pool. Update env default, serve-script fallback, and the deployment
doc line.
This commit is contained in:
Frank
2026-08-18 08:01:17 -07:00
parent 819b9d448a
commit a08be1641f
3 changed files with 12 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/deployment.md
+3 -2
View File
@@ -120,8 +120,9 @@ The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
companions) under the alias `frank-default-large` on port 52625. It runs
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
(`--spec-type draft-mtp`), f16 KV for both K and V, context 262144 across
2 parallel slots, batch 2048 / ubatch 2048, 32 context checkpoints, 8 GiB
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context per
slot (524288 total across 2 parallel slots), batch 2048 / ubatch 2048, 32
context checkpoints, 8 GiB
prompt cache, and reasoning in deepseek format. The unit file is
`scripts/frank-default-large.service`. It launches through
`scripts/serve-frank-large.sh`, which reads tunables from