From 1fee99c8ce3f36c1eeef0fae981b8a9530481b34 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 5 Oct 2026 05:55:39 -0700 Subject: [PATCH] docs(frank-primary): record the verified 2026-10-05 prefix-cache-off deploy Deploy b820d34, measured: --no-enable-prefix-caching in the engine argv, health 200, warm boot 76.57 s (compile cache intact), expert plan unchanged (17.77 GiB / 7,762 slots), GPU KV cache 424,678 tokens (up from 401,408 at the same byte ask - pool now prices ~7,456 B/token with the per-block mamba checkpoint pages gone; still clears the 400,700 floor), zero gdn_hip lines, and the 40,771-token boundary probe (13 x 3136 = 40,768 crossed) decoded the 1-400 count coherently (0.997 monotone, tail ...398 399 400). e2e 32/33 - the failure is the admin spec's FrankBook-Pro leg (mini asleep), environmental. --- ...server-qwen38-fp8-expert-offload-tuning.md | 30 ++++++++++++++----- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md b/serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md index b3954b3..02a06a5 100644 --- a/serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md +++ b/serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md @@ -350,13 +350,29 @@ no prefix reuse — every turn re-prefills the whole history instead of at most 3135 tail tokens. A silently corrupted conversation is worse than a slower first token. -**Acceptance (2026-10-05 deploy):** boot healthy with the expert plan -unchanged (arena 17.77 GiB / 7,762 slots — the flag touches no plan -term), "GPU KV cache size" still clears 400,700 tokens (if the -align-mode block-size raise disappears the mamba per-token price moves -and the pool may need `FRANK_PRIMARY_KV_BYTES` up a notch), zero -`gdn_hip` lines, and one request whose context crosses 40,768 tokens -in-session decodes coherently. +**Verified 2026-10-05 (deploy b820d34):** + +- Engine argv carries `--no-enable-prefix-caching`; health 200; warm boot + `init engine … took 76.57 s` (the mounted compile cache is intact). +- Boot plan unchanged, as predicted: expert cache **17.77 GiB / 7,762 + slots** (the flag touches no plan term), CUDA-graph estimate 0.07 GiB + vs 0.08 GiB captured (inside the 12 MiB/graph charge). +- `GPU KV cache size: 424,678 tokens` — **up** from 401,408 at the same + 3,166,500,000 B ask: the pool now prices ~7,456 B/token (was 7,891.8), + the mamba/side per-token term dropping once the per-block checkpoint + pages are gone; the 3136 attention block-size raise nonetheless stays + (the page-size invariant is enforced in mode "none" too). Frank's + 400,700-token floor is cleared with ~24k of headroom. +- **Zero `gdn_hip` lines** in the boot-and-traffic window (only the two + INFO module banners mention the name). +- Boundary probe (single request, no prefix reuse): prompt 39,280 + + completion 1,491 = **40,771 total tokens — across the 40,768 line**; + the task (write 1–400, one per line) completed coherently: 399 integers + parsed, 0.997 monotone, tail `… 398 399 400`. Pre-fix, this boundary is + exactly where the conversation degraded (the 13th 3136-page re-seed). +- e2e: 32/33 pass; the one failure is the admin spec's + 192.168.0.112:8989 leg with FrankBook-Pro asleep (environmental, + unrelated to this change; re-run `tests/e2e/run.sh` once the mini is up). ### Shape F/G (2026-10-03): where the free VRAM actually is, and what forcing it cost