docs(vault): replace an invented --checkpoint-policy name with the three that exist
The replay law named `--checkpoint-policy floor`; config.cpp admits exactly
geometric-backoff (default), tip-only, keep-all and refuses anything else at
boot, so the recorded arm could never be deployed. Replaced with the real
semantics, priced from core/mem/prefix.{h,cpp}: the default keeps a geometric
ladder (9 snapshots for a 200k session at interval 2048 -- exactly the live
ledger line -- so a turn diverging d behind the tip can resume at tip-3d:
diverge at 120k, replay 21.7k); retention is not the write path, so keep-all
buys a bounded replay for pinned host RAM alone (196 slots x 111.98 MiB = 21.4
GiB against the store's eighth-of-host budget of 31.5 GiB, hard bound = room);
and shortening --checkpoint-interval cannot fix the deep case -- the coarsest
ladder scale is interval x 2^k under the depth, so the deepest gap stays about
half the prompt at every interval. §9 keeps only the three-name locator.
This commit is contained in:
1 parent
9f7457e39e
commit
b1fc1113c6
2 files changed
+20
-9
No files matched your search
@@ -277,7 +277,9 @@ Three instruments the PromQL above cannot reach:
|
||||
(linear + paged) model a prefix hit reuses only as far as the last **linear-state** checkpoint, so
|
||||
cached tokens are replayed as prefill and stall decoders for it; frank-archon ran 38 % of all its
|
||||
prefill tokens that way, every `prefix_hit_cuts_total` labelled `reason="no_snapshot"`. A high
|
||||
prefill rate is not always a client that sends new tokens.
|
||||
prefill rate is not always a client that sends new tokens. Retention is `--checkpoint-policy` -
|
||||
exactly three names, `geometric-backoff` (default), `tip-only`, `keep-all`, one owner:
|
||||
[[frank-archon]].
|
||||
- **Bucket the arriving request too** — `histogram_quantile` over
|
||||
`vllm:time_to_first_token_seconds_bucket` by the same prefill series. The decode gauge only sees
|
||||
sequences that are already running; the one waiting for its first token pays the same steps and
|
||||
|
||||
Reference in new issue
Block a user