docs(frank-primary): record the verified 2026-10-05 prefix-cache-off deploy

Deploy b820d34, measured: --no-enable-prefix-caching in the engine argv,
health 200, warm boot 76.57 s (compile cache intact), expert plan
unchanged (17.77 GiB / 7,762 slots), GPU KV cache 424,678 tokens (up
from 401,408 at the same byte ask - pool now prices ~7,456 B/token with
the per-block mamba checkpoint pages gone; still clears the 400,700
floor), zero gdn_hip lines, and the 40,771-token boundary probe
(13 x 3136 = 40,768 crossed) decoded the 1-400 count coherently
(0.997 monotone, tail ...398 399 400). e2e 32/33 - the failure is the
admin spec's FrankBook-Pro leg (mini asleep), environmental.
This commit is contained in:
Frank
2026-10-05 05:55:39 -07:00
parent b820d341f5
commit 1fee99c8ce
@@ -350,13 +350,29 @@ no prefix reuse — every turn re-prefills the whole history instead of at
most 3135 tail tokens. A silently corrupted conversation is worse than a
slower first token.
**Acceptance (2026-10-05 deploy):** boot healthy with the expert plan
unchanged (arena 17.77 GiB / 7,762 slots — the flag touches no plan
term), "GPU KV cache size" still clears 400,700 tokens (if the
align-mode block-size raise disappears the mamba per-token price moves
and the pool may need `FRANK_PRIMARY_KV_BYTES` up a notch), zero
`gdn_hip` lines, and one request whose context crosses 40,768 tokens
in-session decodes coherently.
**Verified 2026-10-05 (deploy b820d34):**
- Engine argv carries `--no-enable-prefix-caching`; health 200; warm boot
`init engine … took 76.57 s` (the mounted compile cache is intact).
- Boot plan unchanged, as predicted: expert cache **17.77 GiB / 7,762
slots** (the flag touches no plan term), CUDA-graph estimate 0.07 GiB
vs 0.08 GiB captured (inside the 12 MiB/graph charge).
- `GPU KV cache size: 424,678 tokens` — **up** from 401,408 at the same
3,166,500,000 B ask: the pool now prices ~7,456 B/token (was 7,891.8),
the mamba/side per-token term dropping once the per-block checkpoint
pages are gone; the 3136 attention block-size raise nonetheless stays
(the page-size invariant is enforced in mode "none" too). Frank's
400,700-token floor is cleared with ~24k of headroom.
- **Zero `gdn_hip` lines** in the boot-and-traffic window (only the two
INFO module banners mention the name).
- Boundary probe (single request, no prefix reuse): prompt 39,280 +
completion 1,491 = **40,771 total tokens — across the 40,768 line**;
the task (write 1–400, one per line) completed coherently: 399 integers
parsed, 0.997 monotone, tail `… 398 399 400`. Pre-fix, this boundary is
exactly where the conversation degraded (the 13th 3136-page re-seed).
- e2e: 32/33 pass; the one failure is the admin spec's
192.168.0.112:8989 leg with FrankBook-Pro asleep (environmental,
unrelated to this change; re-run `tests/e2e/run.sh` once the mini is up).
### Shape F/G (2026-10-03): where the free VRAM actually is, and what forcing it cost