docs(fleet): halogen owns no timing instruments; warm-cache cost measured after the pool raise

Post-deploy probe on frank-dragoon: one 101,987-token prompt through
/v1/chat/completions took 81.5 s cold (1,251 t/s of prefill) and 0.3 s on two
repeats — HALOGEN_CACHE_INPLACE=1 keeps the KV rows in the pool, so doubling the
pool evicted nothing the prompt cache needed. The RAM price is still real:
MemAvailable 26 -> 22 GiB.

Census of the same engine's surface corrected the Benchmarking Policy, which
quoted llama.cpp instruments at a halogen endpoint: frank-dragoon answers
/health, /metrics, /v1/models and POST /v1/chat/completions and 404s /ping,
/props, /completion and /tokenize, and no response carries Server-Timing. The
/completion `timings` object and the ttft;dur= header are frank-zealot's; on
frank-dragoon the instrument is wall clock plus usage.prompt_tokens.
This commit is contained in:
Frank
2026-10-18 13:36:35 -07:00
parent c0759492e6
commit e7841bf9d6
3 changed files with 43 additions and 13 deletions
+15 -4
View File
@@ -2200,10 +2200,13 @@ every restart since the cutover.
to 200704`, and `62.9 GiB of weights locked … 11.0 GiB of KV pool, 5.7 GiB of
working memory, 79.6 GiB in all`; prompt cache ON (12 entries × 111 MiB at
ctx 200704 = 1.3 GiB), n-gram 47.68 GiB still `Pin::None`, no clamp line.
- The price is exactly the FLAGS.md collision: host `MemAvailable` fell
**26 → 22 GiB**, because every pool position is RAM the n-gram table's page
cache loses. The warm prompt-cache ratio (136.7× measured pre-raise) was NOT
re-measured — recorded in the runbook as the number to watch.
- Warm reuse survived the raise, measured right after the deploy: one
**101,987**-token wikitext prompt through `/v1/chat/completions` took
**81.5 s** cold (1,251 t/s of prefill) and **0.3 s** on each of two repeats —
`HALOGEN_CACHE_INPLACE=1` keeps the KV rows in the pool, so a warm turn
prefills nothing and the doubling evicted nothing the cache needed. The RAM
price is still real: host `MemAvailable` fell **26 → 22 GiB**, because every
pool position is RAM the n-gram table's page cache loses.
- Every document quoted **~10.8 GiB**, the ~28 KiB/position arithmetic; the
engine reserves **11.0 GiB** and its preflight bills **11.3**. Re-keyed in
AGENTS.md (topology row, KV-pool law, memory ledger), the `frank-dragoon.env`
@@ -2226,3 +2229,11 @@ every restart since the cutover.
200,704 → **262,144** with the 9B description; dragoon 262,144 → **200,704**,
which had kept the pre-lowering window while its own description said
200,704.
- Census of the halogen HTTP surface while doing it, because the fleet bench
law quoted the wrong engine: frank-dragoon answers **`/health`, `/metrics`,
`/v1/models`, `/v1/chat/completions`(POST)** and 404s `/ping`, `/props`,
`/completion`, `/tokenize`; the chat response carries **no `Server-Timing`**
header (`server: uvicorn`, only `date`/`content-*`). So the `/completion`
`timings` object and the `ttft;dur=` header are **llama.cpp** instruments —
frank-zealot's, not dragoon's. AGENTS.md's Benchmarking Policy re-keyed to
say which engine owns which instrument.