docs: point the frank-medium runbook and observability page at the new scrape
The capacity work in this runbook was done with ad-hoc probes; the same signals now have history in Prometheus, so the page says which series to watch and admits the honest gap — this stack has no alerting, so a saturated pool shows up on a graph after the fact rather than paging anyone.
This commit is contained in:
@@ -409,6 +409,18 @@ co-tenants evict each other turn by turn whatever the concurrency limit is. A
|
||||
card can hold two large families' caches, or one cache plus a live generation,
|
||||
but not both. That is the wall, and it is capacity, not scheduling.
|
||||
|
||||
### Watching it instead of measuring it by hand
|
||||
|
||||
Everything above was diagnosed with ad-hoc probes. The same signals are now
|
||||
scraped — Prometheus has `frank-medium` and `frank-large` jobs
|
||||
(`shared/prometheus/prometheus.yml`), so the numbers that matter have history:
|
||||
`vllm:kv_cache_usage_perc` (per-card pool fill), `vllm:num_requests_waiting`
|
||||
(queue depth), `rate(vllm:prefix_cache_hits_total[5m]) /
|
||||
rate(vllm:prefix_cache_queries_total[5m])` (whether pinned conversations are
|
||||
served warm), and the request-latency histograms. See
|
||||
[[Observability on Dankclaw]]. There is still **no alerting** in this stack, so
|
||||
a saturated pool will not page anyone — it shows up on a graph after the fact.
|
||||
|
||||
## Citations
|
||||
|
||||
- `ROCm/clr` `rocclr/device/rocm/rocvirtual.cpp:3963` — the hostcall-buffer
|
||||
|
||||
Reference in New Issue
Block a user