observability: name the prefix-cache series that actually exist
vllm:prefix_cache_hit_rate does not exist in this build; the exported pair is vllm:prefix_cache_hits_total / vllm:prefix_cache_queries_total, whose rates give the hit rate. A comment naming a series nobody can query is worse than no comment.
This commit is contained in:
1 parent
42b47063ec
commit
0f291115bd
1 file changed
+5
-3
@@ -57,9 +57,11 @@ scrape_configs:
|
|||||||
# LLM fleet. Scraped on the LAN addresses, not the guarded vhosts: these are
|
# LLM fleet. Scraped on the LAN addresses, not the guarded vhosts: these are
|
||||||
# machine clients and Prometheus cannot solve an Anubis challenge.
|
# machine clients and Prometheus cannot solve an Anubis challenge.
|
||||||
# frank-medium is vLLM, and its series are the ones that explain behaviour
|
# frank-medium is vLLM, and its series are the ones that explain behaviour
|
||||||
# under load — vllm:kv_cache_usage_perc (how full the pool is),
|
# under load — vllm:kv_cache_usage_perc (how full the per-card pool is),
|
||||||
# vllm:prefix_cache_hit_rate (whether pinned conversations are being reused),
|
# vllm:num_requests_waiting (queue depth), the request-latency histograms,
|
||||||
# vllm:num_requests_waiting (queue depth), and the request-latency histograms.
|
# and the prefix-cache pair vllm:prefix_cache_hits_total over
|
||||||
|
# vllm:prefix_cache_queries_total (their rates give the hit rate, which says
|
||||||
|
# whether pinned conversations are actually being reused).
|
||||||
# frank-large is llama.cpp and exposes throughput counters under llamacpp:.
|
# frank-large is llama.cpp and exposes throughput counters under llamacpp:.
|
||||||
- job_name: "frank-medium"
|
- job_name: "frank-medium"
|
||||||
metrics_path: "/metrics"
|
metrics_path: "/metrics"
|
||||||
|
|||||||
Reference in new issue
Block a user