observability: name the prefix-cache series that actually exist
vllm:prefix_cache_hit_rate does not exist in this build; the exported pair is vllm:prefix_cache_hits_total / vllm:prefix_cache_queries_total, whose rates give the hit rate. A comment naming a series nobody can query is worse than no comment.
This commit is contained in:
@@ -57,9 +57,11 @@ scrape_configs:
|
||||
# LLM fleet. Scraped on the LAN addresses, not the guarded vhosts: these are
|
||||
# machine clients and Prometheus cannot solve an Anubis challenge.
|
||||
# frank-medium is vLLM, and its series are the ones that explain behaviour
|
||||
# under load — vllm:kv_cache_usage_perc (how full the pool is),
|
||||
# vllm:prefix_cache_hit_rate (whether pinned conversations are being reused),
|
||||
# vllm:num_requests_waiting (queue depth), and the request-latency histograms.
|
||||
# under load — vllm:kv_cache_usage_perc (how full the per-card pool is),
|
||||
# vllm:num_requests_waiting (queue depth), the request-latency histograms,
|
||||
# and the prefix-cache pair vllm:prefix_cache_hits_total over
|
||||
# vllm:prefix_cache_queries_total (their rates give the hit rate, which says
|
||||
# whether pinned conversations are actually being reused).
|
||||
# frank-large is llama.cpp and exposes throughput counters under llamacpp:.
|
||||
- job_name: "frank-medium"
|
||||
metrics_path: "/metrics"
|
||||
|
||||
Reference in New Issue
Block a user