diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index 5c55db8..f65eb52 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -941,3 +941,31 @@ timestamp: 2026-09-14 `ps`/`ss` and prove it abandoned (frozen CPU ticks, no conns) before reclaiming. Scratch servers on loopback are invisible to the endpoint monitor, so they strand the fleet instead of failing loudly. + +## 2026-09-18 (afternoon) — frank-medium: the host-RAM prompt cache doubles to 16 GiB + +- **The old cap was binding, not theoretical.** `--cache-ram` caps the total + bytes of saved prompt states in HOST RAM (`server_prompt_cache`), and + `alloc()` refuses any single state larger than the whole cap with one `WRN`. + The journal held **137** such refusals in the two days from 2026-09-16 — + `prompt state size 11388.990 MiB exceeds cache size limit 8192.000 MiB, + skipping` — i.e. every conversation past ~170k tokens was uncacheable, and + its next turn re-prefilled the whole thing. +- **The cap is a token budget in disguise.** A state is the serialized KV of a + whole sequence plus its constant-size DeltaNet state plus the MTP draft + context's own KV: 16 full-attention layers x 4 kv heads x 256 dim x 2 B x 2 + (K,V) = 64 KiB per token, plus 4096 B for the draft context. So 8192 MiB held + ~120k tokens and 16384 MiB holds ~240k — and a FULL 196608-token slot state + (~12.8 GiB), which the old cap refused outright, now fits with ~3 GiB spare. +- **Deployed change** (`1acca43`): `CACHE_RAM` 8192 → 16384 in + `scripts/frank-medium.env`, with the law and the journal evidence in the + file. Host RAM, allocated on demand and never pinned, so no VRAM number + moves; bounded rather than `-1` because dankserver shares its 62 GiB with the + media-stack co-tenants. +- **New seam** (`tests/frank_medium_unit.bats`): the cap must clear one full + slot's state and may not claim more than half the host's RAM. It fails on the + old value (8192 < 13056 MiB) and passes on the new one — checked both ways. +- **Adjacent correction:** the same block claimed 0 was the build's default for + `--ctx-checkpoints`; this binary's `--help` says **32**, so the 0 is a + deliberate deviation. Fixed in the env comment and in the runbook's flag + table.