dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-medium: checkpoints back to the build's default (0)

Browse Source 
Step 2 of the operator's six. --ctx-checkpoints 8 held per-slot KV snapshots;
0 is what llama.cpp ships, and the fleet's clients open conversations fresh.
This commit is contained in:
Frank committed 2026-09-16 19:51:06 -07:00
1 parent b6d4914fcc
commit 169eba0734
1 file changed
+4 -1
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-medium.env
+4 -1
View File
@@ -129,7 +129,10 @@ THREAD_BATCH=8
# f16 K/V: an agentic checkpoint is the wrong place to spend KV precision. # f16 K/V: an agentic checkpoint is the wrong place to spend KV precision.
CACHE_TYPE_K=f16 CACHE_TYPE_K=f16
CACHE_TYPE_V=f16 CACHE_TYPE_V=f16
CTX_CHECKPOINTS=8 # Context checkpoints: 0 is the build's default -- no per-slot KV snapshots, so
# a re-sent or rewound conversation re-prefills instead of rewinding. Stated
# explicitly so the tunable stays visible rather than merely absent.
CTX_CHECKPOINTS=0
CACHE_RAM=8192 CACHE_RAM=8192
# Sampling defaults for requests that send none -- the values the vLLM # Sampling defaults for requests that send none -- the values the vLLM