docs(frank-primary): record the verified 2026-10-05 prefix-cache-off deploy
Deploy b820d34, measured: --no-enable-prefix-caching in the engine argv,
health 200, warm boot 76.57 s (compile cache intact), expert plan
unchanged (17.77 GiB / 7,762 slots), GPU KV cache 424,678 tokens (up
from 401,408 at the same byte ask - pool now prices ~7,456 B/token with
the per-block mamba checkpoint pages gone; still clears the 400,700
floor), zero gdn_hip lines, and the 40,771-token boundary probe
(13 x 3136 = 40,768 crossed) decoded the 1-400 count coherently
(0.997 monotone, tail ...398 399 400). e2e 32/33 - the failure is the
admin spec's FrankBook-Pro leg (mini asleep), environmental.
This commit is contained in:
@@ -350,13 +350,29 @@ no prefix reuse — every turn re-prefills the whole history instead of at
|
||||
most 3135 tail tokens. A silently corrupted conversation is worse than a
|
||||
slower first token.
|
||||
|
||||
**Acceptance (2026-10-05 deploy):** boot healthy with the expert plan
|
||||
unchanged (arena 17.77 GiB / 7,762 slots — the flag touches no plan
|
||||
term), "GPU KV cache size" still clears 400,700 tokens (if the
|
||||
align-mode block-size raise disappears the mamba per-token price moves
|
||||
and the pool may need `FRANK_PRIMARY_KV_BYTES` up a notch), zero
|
||||
`gdn_hip` lines, and one request whose context crosses 40,768 tokens
|
||||
in-session decodes coherently.
|
||||
**Verified 2026-10-05 (deploy b820d34):**
|
||||
|
||||
- Engine argv carries `--no-enable-prefix-caching`; health 200; warm boot
|
||||
`init engine … took 76.57 s` (the mounted compile cache is intact).
|
||||
- Boot plan unchanged, as predicted: expert cache **17.77 GiB / 7,762
|
||||
slots** (the flag touches no plan term), CUDA-graph estimate 0.07 GiB
|
||||
vs 0.08 GiB captured (inside the 12 MiB/graph charge).
|
||||
- `GPU KV cache size: 424,678 tokens` — **up** from 401,408 at the same
|
||||
3,166,500,000 B ask: the pool now prices ~7,456 B/token (was 7,891.8),
|
||||
the mamba/side per-token term dropping once the per-block checkpoint
|
||||
pages are gone; the 3136 attention block-size raise nonetheless stays
|
||||
(the page-size invariant is enforced in mode "none" too). Frank's
|
||||
400,700-token floor is cleared with ~24k of headroom.
|
||||
- **Zero `gdn_hip` lines** in the boot-and-traffic window (only the two
|
||||
INFO module banners mention the name).
|
||||
- Boundary probe (single request, no prefix reuse): prompt 39,280 +
|
||||
completion 1,491 = **40,771 total tokens — across the 40,768 line**;
|
||||
the task (write 1–400, one per line) completed coherently: 399 integers
|
||||
parsed, 0.997 monotone, tail `… 398 399 400`. Pre-fix, this boundary is
|
||||
exactly where the conversation degraded (the 13th 3136-page re-seed).
|
||||
- e2e: 32/33 pass; the one failure is the admin spec's
|
||||
192.168.0.112:8989 leg with FrankBook-Pro asleep (environmental,
|
||||
unrelated to this change; re-run `tests/e2e/run.sh` once the mini is up).
|
||||
|
||||
### Shape F/G (2026-10-03): where the free VRAM actually is, and what forcing it cost
|
||||
|
||||
|
||||
Reference in New Issue
Block a user