docs(vault): replace an invented --checkpoint-policy name with the three that exist

The replay law named `--checkpoint-policy floor`; config.cpp admits exactly
geometric-backoff (default), tip-only, keep-all and refuses anything else at
boot, so the recorded arm could never be deployed. Replaced with the real
semantics, priced from core/mem/prefix.{h,cpp}: the default keeps a geometric
ladder (9 snapshots for a 200k session at interval 2048 -- exactly the live
ledger line -- so a turn diverging d behind the tip can resume at tip-3d:
diverge at 120k, replay 21.7k); retention is not the write path, so keep-all
buys a bounded replay for pinned host RAM alone (196 slots x 111.98 MiB = 21.4
GiB against the store's eighth-of-host budget of 31.5 GiB, hard bound = room);
and shortening --checkpoint-interval cannot fix the deep case -- the coarsest
ladder scale is interval x 2^k under the depth, so the deepest gap stays about
half the prompt at every interval. §9 keeps only the three-name locator.
This commit is contained in:
Frank committed 2026-10-20 22:34:13 -07:00
1 parent 9f7457e39e
commit b1fc1113c6
2 files changed
+20 -9

No files matched your search

@@ -277,7 +277,9 @@ Three instruments the PromQL above cannot reach:
(linear + paged) model a prefix hit reuses only as far as the last **linear-state** checkpoint, so
cached tokens are replayed as prefill and stall decoders for it; frank-archon ran 38 % of all its
prefill tokens that way, every `prefix_hit_cuts_total` labelled `reason="no_snapshot"`. A high
prefill rate is not always a client that sends new tokens.
prefill rate is not always a client that sends new tokens. Retention is `--checkpoint-policy` -
exactly three names, `geometric-backoff` (default), `tip-only`, `keep-all`, one owner:
[[frank-archon]].
- **Bucket the arriving request too** — `histogram_quantile` over
`vllm:time_to_first_token_seconds_bucket` by the same prefill series. The decode gauge only sees
sequences that are already running; the one waiting for its first token pays the same steps and