frank-primary: enable prefix caching (E8 qualified: 9x-63x reuse TTFT, zero VRAM cost, cold path unchanged)
This commit is contained in:
1 parent
92579b0674
commit
41e918929f
1 file changed
+6
-1
@@ -44,4 +44,9 @@ R9V_FULL_MUTABLE_CACHE=0
|
||||
# Braces in values get the same single-quote law as parentheses (source
|
||||
# parses assignment values).
|
||||
R9V_CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/qwen-sharp-v22.4.0.jinja
|
||||
R9V_CHAT_TEMPLATE_KWARGS='{"preserve_thinking":true}'
|
||||
R9V_CHAT_TEMPLATE_KWARGS='{"preserve_thinking":true}'
|
||||
# Prefix caching (bench matrix E8, 2026-09-28): qualified on the q8 profile
|
||||
# at 131072, zero VRAM cost (reuses freed KV blocks), cold path unchanged.
|
||||
# Reuse-heavy traffic (harness retries/resends) hits from the third round:
|
||||
# 32k TTFT 44.2 s -> 3.6 s, 131k 439 s -> 6.9 s. Reversible: flip to 0.
|
||||
R9V_ENABLE_PREFIX_CACHING=1
|
||||
Reference in new issue
Block a user