dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

fix: frank-default-small ctx-size 786432 (3 slots x 262144 each)

Browse Source 
llama.cpp splits --ctx-size across --parallel slots, so 262144 total gave
87552 per stream. Non-negotiable is 262144 per stream: total pool must be
786432.
This commit is contained in:
Frank committed 2026-08-11 01:05:24 -07:00
1 parent f93096f2bf
commit 6a86a9df93
1 file changed
+2 -1
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-default-small.env
+2 -1
View File
@@ -4,7 +4,8 @@ MODEL_DIR=/home/dank/models
ALIAS=frank-default-small ALIAS=frank-default-small
HOST=0.0.0.0 HOST=0.0.0.0
PORT=7770 PORT=7770
CTX_SIZE=262144 # 3 parallel slots x 262144 ctx each = 786432 total context pool.
CTX_SIZE=786432
PARALLEL=3 PARALLEL=3
SPEC_TYPE=draft-dflash SPEC_TYPE=draft-dflash
SPEC_N_MAX=7 SPEC_N_MAX=7