frank-default-large: full native 262144 context per slot (524288 total)

llama.cpp splits -c across -np slots; to give each of the 2 slots the full
262144 native context, set CTX_SIZE=524288. KV ~32 GiB; fits the 96 GiB
unified pool. Update env default, serve-script fallback, and the deployment
doc line.
This commit is contained in:
Frank committed 2026-08-18 08:01:17 -07:00
1 parent 819b9d448a
commit a08be1641f
3 files changed
+12 -9

No files matched your search

+1 -1
View File
@@ -37,7 +37,7 @@ fi
: "${ALIAS:=frank-default-large}"
: "${BIND_ADDR:=0.0.0.0}"
: "${LISTEN_PORT:=52625}"
: "${CTX_SIZE:=262144}"
: "${CTX_SIZE:=524288}"
: "${PARALLEL:=2}"
: "${BATCH_SIZE:=2048}"
: "${UBATCH_SIZE:=2048}"