dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

frank-default-large: 200000 context per slot (400000 total, 2 slots)

Browse Source 
User request 2026-08-19: lower per-stream context from 262144 to 200000.
CTX_SIZE is the total; llama.cpp splits it across the 2 parallel slots.
f16 KV drops from ~32 GiB to ~24 GiB, adding OOM headroom.

Docs: sync the 2026-08-18 service swap (frank-default-large on dankclaw,
frank-default-small on dankserver) across AGENTS.md, README.md,
deployment.md, ciru-activefpx-promptforge.md, and index.md. The pi
models.json contextWindow/maxTokens for frank-default-large move
262144 -> 200000 (separate file, outside the repo).
This commit is contained in:
Frank
2026-08-19 10:38:53 -07:00
parent 9fd82dc6e1
commit 8e4d63b68d
6 changed files with 32 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/deployment.md
+3 -3
View File
@@ -9,7 +9,7 @@ Two servers run LLM inference services. Each server pulls its scripts from this
| Target | Host | Runtime | Service | Port |
|--------|------|---------|---------|------|
| Dankserver | dank@dankserver | upstream llama.cpp (Qwen3.6-35B-A3B Q6_K_P + ngram-mod, MoE on CPU) | frank-default-small | 7770 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP + PromptForge DualView, 256K ctx / 2 slots) | frank-default-large | 52625 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP + PromptForge DualView, 200K ctx / 2 slots) | frank-default-large | 52625 |
## Workflow
@@ -120,8 +120,8 @@ The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
companions) under the alias `frank-default-large` on port 52625. It runs
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context per
slot (524288 total across 2 parallel slots), batch 2048 / ubatch 2048, 32
(`--spec-type draft-mtp`), f16 KV for both K and V, 200000 context per
slot (400000 total across 2 parallel slots), batch 2048 / ubatch 2048, 16
context checkpoints, 8 GiB
prompt cache, and reasoning in deepseek format. The unit file is
`scripts/frank-default-large.service`. It launches through