docs: prefix caching deployed on frank-primary (E8-qualified, 9x/63x reuse TTFT)

serverAppConfigs-docs/LLM/frank-primary-r9v-radiance-qwen38-q8.md

@@ -40,7 +40,7 @@ The repo carries a **thin** wiring; the engine shape lives on-host:
 | Piece | File | Law |
 |---|---|---|
 | Unit | `scripts/frank-primary.service` | Root unit, `User=dank`, `LimitMEMLOCK=infinity`, `ExecStop=podman stop -t 120 frank-primary-r9v`, `Restart=on-failure` |
-| Env | `scripts/frank-primary.env` | Production overrides ONLY: bind 0.0.0.0, :7770, alias `frank-primary`, container `frank-primary-r9v`, GPU BDFs, dev-overlay adapter, `R9V_FULL_MUTABLE_CACHE=0`, sharp chat template |
+| Env | `scripts/frank-primary.env` | Production overrides ONLY: bind 0.0.0.0, :7770, alias `frank-primary`, container `frank-primary-r9v`, GPU BDFs, dev-overlay adapter, `R9V_FULL_MUTABLE_CACHE=0`, `R9V_PLE_READY_TIMEOUT=1800`, prefix caching ON, sharp chat template |
 | Serve | `scripts/serve-frank-primary.sh` | Layers setup state under repo env → execs the fork's own `scripts/launch.sh` via the docker→podman shim → `docker wait` |
 | Deployer | `scripts/deploy-remote.sh` `deploy_llm_primary` | Gates on setup state ready, image digest, PLE bytes, all six shard files + mmproj; MainPID false-green check |
 
@@ -140,9 +140,11 @@ edited in place — mutating it drifts the qualification.
   ssd-mmap residency).
 - Context envelope: `R9V_MAX_MODEL_LEN=131072`; harness `models.yml`
   carries `contextWindow: 131072`, `maxTokens: 36864` (compaction-gap
-  math: 70% × 131,072 = 91,750 + 36,864 = 128,614 ≤ 131,072).
+  math: 70% × 131,072 = 91,750 + 36,864 = 128,614 ≤ 131,072). The KV
-- Prefix caching OFF (`R9V_ENABLE_PREFIX_CACHING=0`), CACHE192 off — the
+  envelope at 131,072 is smaller than the uncensored 196,608 shape.
-  KV envelope at 131,072 is smaller than the uncensored 196,608 shape.
+- Prefix caching OFF at cutover (`R9V_ENABLE_PREFIX_CACHING=0`), CACHE192
+  off — superseded 2026-09-28: prefix caching is ON (see bench matrix
+  below).
 
 ## Prefill / decode bench matrix (2026-09-28, scratch port 7773)
 
@@ -183,9 +185,10 @@ prompt TTFT 44.2 s → **3.6–4.3 s** (9x), 131k 439 s → **6.9 s** (63x).
 Hit pattern: rounds 1–2 miss, round 3+ hit (identical-prompt and
 conversation-growth probes both). For the harness's resend-every-turn
 traffic that is a ~40–440 s TTFT win per steady-state turn; the one-turn
-miss costs nothing extra. **RECOMMENDED for production — pending Frank's
+miss costs nothing extra. **DEPLOYED 2026-09-28** (Frank's go-ahead): the
-go-ahead** (one env line; the launcher default is already ON, the q8 state
+repo env sets `R9V_ENABLE_PREFIX_CACHING=1` (commit `41e9189`), verified
-opted out).
+on :7770 (`--enable-prefix-caching` in the container argv, health 200,
+live completion answering).
 
 ## Ops
 

serverAppConfigs-docs/log.md

@@ -1,6 +1,13 @@
 # Bundle Change Log
 
+
 ## 2026-09-28
+- `scripts/frank-primary.env`, runbook — **prefix caching DEPLOYED on
+  frank-primary** (Frank's go-ahead; commit `41e9189`): E8-qualified, zero
+  VRAM cost, 9x/63x reuse TTFT. Verified on :7770 (argv flag, health 200,
+  completion "PREFIX-CACHE-LIVE"). Gotcha re-confirmed: the container is
+  in dank's ROOTLESS podman store (`sudo podman` inspects an empty store
+  and reports nothing); probe via the shim's `docker` or the unit MainPID.
 - `/scripts/frank-primary.env`, `/serverAppConfigs-docs/LLM/frank-primary-r9v-radiance-qwen38-q8.md`, `/serverAppConfigs-docs/index.md` — **frank-primary prefill/decode bench matrix** (scratch :7773, prod stopped 00:06 for the window per fleet bench law): baseline E0 (chunk 4096 / 80-slot sync / MTP4) = PP8K 954.6, PP131K 883.0 tok/s, decode 141.9 ms/step ≈ 23.8 t/s effective. Chunk 8192 (E1R): PP8K +25.9% (1201.4) but decode −8.5% and PP131K −4.4% — short-prompt win, decode tax, not the fleet's dominant shape. Chunk 16384 boots but OOMs at request time (0.93 GiB free) — 4096 is the ceiling (engine spec-config cap + memory). Expert cache 80 slots is the policy max (doctor: capacities 48/64/80, synchronous LRU only — async unsupported, 96/112 rejected); MTP depth 6 is 3.7x SLOWER than depth 4. Prefix caching (E8/E9): zero VRAM cost, cold path unchanged, reuse hits 32k 44.2→3.6–4.3 s (9x), 131k 439→6.9 s (63x); hit pattern rounds 1–2 miss / 3+ hit — RECOMMENDED, pending Frank's go-ahead. Two failure laws recorded: unqualified shapes fail mid-request (health stays 200), and PLE registration can blow the engine's 600 s `VLLM_PLE_OFFLOAD_READY_TIMEOUT` under page-cache churn (measured 740 s) — launch.sh gained `R9V_PLE_READY_TIMEOUT` forwarding (fork `2de0357`, local-only) and prod now sets 1800.
 - **Incident — frank-primary down 00:06 → ~11:00 (11h)**: the bench window's required prod stop was never restored — the matrix + follow-ups ran 00:44 → 09:02 and the session let the endpoint sit dead (the 2026-09-18 "a bench session owes the fleet a health 200" lesson, re-offended). The first restore boot (09:46) then failed on the PLE 600 s timeout above (unit "Deactivated successfully", no journal error — the serve script exits 0 on the wait). Restored by the `R9V_PLE_READY_TIMEOUT=1800` fix deploy (f187165, from main — the deploy gate refuses non-main): health 200 + MainPID = the serve script's own `podman wait` + live completion verified ~11:00. Deploy e2e gate 32/33 — the one failure (11-admin) needs the FrankBook-Pro Mac mini, which was asleep and ignored two WoL bursts (240 packets); recurring pattern per the wol-wake skill. Lesson: a multi-hour bench window needs an explicit restore task, not a "watch for DONE" — DONE ≠ prod up.