docs(fleet): halogen owns no timing instruments; warm-cache cost measured after the pool raise

Post-deploy probe on frank-dragoon: one 101,987-token prompt through
/v1/chat/completions took 81.5 s cold (1,251 t/s of prefill) and 0.3 s on two
repeats — HALOGEN_CACHE_INPLACE=1 keeps the KV rows in the pool, so doubling the
pool evicted nothing the prompt cache needed. The RAM price is still real:
MemAvailable 26 -> 22 GiB.

Census of the same engine's surface corrected the Benchmarking Policy, which
quoted llama.cpp instruments at a halogen endpoint: frank-dragoon answers
/health, /metrics, /v1/models and POST /v1/chat/completions and 404s /ping,
/props, /completion and /tokenize, and no response carries Server-Timing. The
/completion `timings` object and the ttft;dur= header are frank-zealot's; on
frank-dragoon the instrument is wall clock plus usage.prompt_tokens.
This commit is contained in:
Frank committed 2026-10-18 13:36:35 -07:00
1 parent c0759492e6
commit e7841bf9d6
3 files changed
+43 -13

No files matched your search

@@ -152,9 +152,15 @@ of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`. Prompt cache ON (12
entries × 111 MiB at ctx 200704 = the 1.3 GiB in the KV-budget line), n-gram
47.68 GiB still disk-backed (`Pin::None`), no clamp line. Host `MemAvailable`
fell 26 → 22 GiB across the raise, which is the FLAGS.md collision being paid
in page cache. **Not re-measured since the raise:** the warm prompt-cache ratio
(136.7× on the pre-raise boot) — that is the number to watch if a long warm
turn starts costing full prefill again.
in page cache. **Warm reuse survived it — measured after the deploy.** One
**101,987**-token wikitext prompt through `/v1/chat/completions` took **81.5 s**
cold (1,251 t/s of prefill) and **0.3 s** on each of two repeats:
`HALOGEN_CACHE_INPLACE=1` keeps the KV rows in the pool, so a warm turn prefills
nothing and the doubling evicted nothing the cache needed. (The pre-raise
136.7× figure was a `ttft` reading; this engine sends **no `Server-Timing`**
header and has **no `/completion` path** — `/ping`, `/props`, `/tokenize` are
404 too — so wall-clock around the POST plus `usage.prompt_tokens` is the only
instrument available here.)
## Measured A/B — same battery, both boots (2026-10-17)
@@ -235,7 +241,12 @@ is mid-thought); vision via OpenAI `image_url` content parts only, ≥256 px
probes; `response_format` unsupported; the chat template rejects
`reasoning_effort` outside `xhigh|medium|low` (the `models.yml` fold stays
mandatory); unknown sampling params are a 400; the prompt cache is on by
default, so a warm 100k follow-up costs seconds. See
default, so a warm 100k follow-up costs seconds (measured after the pool raise:
**81.5 s cold vs 0.3 s warm** at 101,987 prompt tokens). The HTTP surface is
thin — `/health`, `/metrics`, `/v1/models` and POST `/v1/chat/completions`
answer, while `/ping`, `/props`, `/completion` and `/tokenize` are 404 and no
response carries `Server-Timing`: llama.cpp's timing instruments do not exist
on this engine, so measure wall-clock plus `usage.prompt_tokens`. See
[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
carries verbatim, and [[deployment]] for the deploy/health gate.