- shared/prometheus/rules/llm-fleet.yml: the llamacpp reuse recording rule
divided cached by prompt, but llama.cpp's prompt_tokens_total EXCLUDES
cached tokens (its own HELP line), so the ratio was unbounded — measured
77 on a fully-warm frank-large, meaning LlmPrefixCacheCollapse could never
fire on that side. Now cached / (cached + prompt), with the traffic floor
counting both terms too; the header documents the unit trap. Also fixed the
LlmEndpointDown description: frank-large's CIRU llama.cpp build DOES export
the llamacpp:* family (verified live), so the alert covers all three
endpoints.
- shared/grafana/provisioning/dashboards/llm-fleet.json: re-cut for two
namespaces. Every panel reads both (the endpoint picker selects); the
KV-pool-fill panel retired 2026-09-16 returned with
vllm:kv_cache_usage_perc; the latency p95 panel returned with the vLLM
histograms (TTFT/ITL/e2e); the alerts panel is scoped to the fleet's
Llm.* rules; the stale halogen-era description is gone. All 21 panel
expressions validated against the live endpoints.
- runbook + log + Observability: prefix caching measured, not inferred —
reuse happens at 1,648-token block granularity (attention block size padded
to the mamba page size); an identical 13,021-token prompt went 2.30 s cold
to 0.68 s warm with 9,888 tokens (76 %) reused, while a 2,521-token prompt
registered zero hits. Rule counts corrected to 6 alerts + 3 recording rules.
./tests/run.sh: 325 pass.
Frank
c42780f9f3