dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-default-small: lower context by 32k per stream for cache headroom

Browse Source 
3 slots x 229376 (262144 - 32768) = 688128 total context. Updates the
.env, models.json (contextWindow/maxTokens 229376 per slot), and the
index/deployment/ornith runbook references.
This commit is contained in:
Frank
2026-08-24 21:50:54 -07:00
parent 4462aa992c
commit 7ba1acdbc5
4 changed files with 8 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-default-small.env
+4 -4
View File
@@ -25,10 +25,10 @@ ALIAS=frank-default-small
BIND_ADDR=0.0.0.0
LISTEN_PORT=7770
# 3 parallel slots x 262144 total context pool (max per-slot), unchanged
# from the prior Qwen3.6 service. Lower PARALLEL if VRAM is tight with the
# larger Q8_0 payload.
CTX_SIZE=786432
# 3 parallel slots x 229376 ctx each (262144 lowered by 32k per stream for
# cache headroom) = 688128 total context pool. The per-slot 229376 matches
# the per-slot context advertised in models.json (229376).
CTX_SIZE=688128
PARALLEL=3
# Spec: MoE model with a bundled MTP head -> native draft-mtp, n_max 4