frank-primary: enable prefix caching (E8 qualified: 9x-63x reuse TTFT, zero VRAM cost, cold path unchanged)

This commit is contained in:
Frank committed 2026-09-28 12:03:56 -07:00
1 parent 92579b0674
commit 41e918929f
1 file changed
+6 -1
+6 -1
View File
@@ -44,4 +44,9 @@ R9V_FULL_MUTABLE_CACHE=0
# Braces in values get the same single-quote law as parentheses (source
# parses assignment values).
R9V_CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/qwen-sharp-v22.4.0.jinja
R9V_CHAT_TEMPLATE_KWARGS='{"preserve_thinking":true}'
R9V_CHAT_TEMPLATE_KWARGS='{"preserve_thinking":true}'
# Prefix caching (bench matrix E8, 2026-09-28): qualified on the q8 profile
# at 131072, zero VRAM cost (reuses freed KV blocks), cold path unchanged.
# Reuse-heavy traffic (harness retries/resends) hits from the third round:
# 32k TTFT 44.2 s -> 3.6 s, 131k 439 s -> 6.9 s. Reversible: flip to 0.
R9V_ENABLE_PREFIX_CACHING=1