dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-default-large: full 262144 context, 1 slot

Browse Source 
User request 2026-08-19 (second pass): parallel slots degraded
performance. PARALLEL=1, CTX_SIZE=262144 (native max, -c equals the
per-slot context with one slot). Concurrent requests queue on the
single slot; the server never preempts a running generation.
pi models.json contextWindow/maxTokens back to 262144 (outside repo).
Docs updated to match.
This commit is contained in:
Frank
2026-08-19 11:17:31 -07:00
parent 8e4d63b68d
commit 048e7eaaba
6 changed files with 22 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -15,7 +15,7 @@ upstream llama.cpp on Dankserver and CIRU ROCmFPX llama.cpp on Dankclaw.
| Service | Host | Runtime | Model | Port |
|---------|------|---------|-------|------|
| frank-default-small | dankserver | llama.cpp (upstream b10121) | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU, 4 slots x ~175K ctx | 7770 |
| frank-default-large | dankclaw | CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK) | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP + PromptForge DualView, 200K ctx / 2 slots | 52625 |
| frank-default-large | dankclaw | CIRU ROCmFPX llama.cpp (ROCm 7.15 SDK) | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP + PromptForge DualView, 262K ctx / 1 slot | 52625 |
The 2026-08-17 redo retired the pipeline-parallel split (PP head + RPC
worker) and the FastFlowLM NPU endpoint. The 2026-08-18 swap retired the