observability: the LLM surfaces follow the two-endpoint split
frank-large has no /metrics to scrape any more, and scraping it would be red by construction: the halogen engine publishes /health, /cache and /v1/* only. That job is removed with the reason written down (the remedy is the blackbox exporter this fleet already wants for the comfyui/services probes), and frank-small's llama.cpp endpoint takes the slot at 192.168.0.143:7771. Homepage: frank-large's card still described the ik runtime, and the new endpoint had no card at all. Both fixed, with a loopback probe -- the checker runs on Dankclaw, which is also what makes the card green-able. Dashboard: the endpoint picker was sourced from a vLLM-only metric, so a llama.cpp job could never be selected, and the prose named frank-large as the llama.cpp endpoint. The picker now matches both runtimes' series and the llama.cpp targets carry the job filter every other panel already used. The seams assert the new contract including the ABSENCE of a 7770 target, so a well-meaning edit cannot re-add a permanently-red probe.
This commit is contained in:
1 parent
43ec77fee6
commit
79cca19919
6 files changed
+57
-22
No files matched your search
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"uid": "llm-fleet",
|
||||
"title": "LLM fleet — capacity and reuse",
|
||||
"description": "The signals that explain frank-medium under load, from the 2026-09-16 capacity work: per-card KV fill, queue depth, prefix-cache reuse, and request latency. frank-large (llama.cpp) has throughput counters only. See serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md for the numbers these panels are read against.",
|
||||
"description": "The signals that explain frank-medium under load, from the 2026-09-16 capacity work: per-card KV fill, queue depth, prefix-cache reuse, and request latency. frank-small (llama.cpp on Dankclaw's RTX 3090) exports throughput counters and appears in the same endpoint picker; frank-large runs the halogen engine, which publishes /health, /cache and /v1/* and no metrics endpoint at all, so it cannot appear here. See serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md for the numbers these panels are read against.",
|
||||
"tags": [
|
||||
"llm",
|
||||
"frank-medium",
|
||||
@@ -26,7 +26,7 @@
|
||||
"uid": "prometheus"
|
||||
},
|
||||
"query": {
|
||||
"query": "label_values(vllm:kv_cache_usage_perc, job)",
|
||||
"query": "label_values({__name__=~\"vllm:kv_cache_usage_perc|llamacpp:.*\"}, job)",
|
||||
"refId": "A"
|
||||
},
|
||||
"includeAll": true,
|
||||
@@ -185,7 +185,7 @@
|
||||
"id": 5,
|
||||
"type": "timeseries",
|
||||
"title": "Decode and prefill throughput",
|
||||
"description": "vLLM generation tokens, and llama.cpp's counters for frank-large (llamacpp:prompt_tokens_seconds is prompt throughput).",
|
||||
"description": "vLLM generation tokens, plus llama.cpp's counters for frank-small: llamacpp:predicted_tokens_seconds is decode and llamacpp:prompt_tokens_seconds is prompt throughput. The llama.cpp series carry the job label of their scrape, so the endpoint picker selects between the runtimes.",
|
||||
"gridPos": {
|
||||
"h": 8,
|
||||
"w": 12,
|
||||
@@ -219,8 +219,13 @@
|
||||
},
|
||||
{
|
||||
"refId": "C",
|
||||
"expr": "llamacpp:predicted_tokens_seconds",
|
||||
"legendFormat": "{{job}} llama.cpp decode"
|
||||
"expr": "llamacpp:predicted_tokens_seconds{job=~\"$job\"}",
|
||||
"legendFormat": "{{job}} llama.cpp decode tok/s"
|
||||
},
|
||||
{
|
||||
"refId": "D",
|
||||
"expr": "llamacpp:prompt_tokens_seconds{job=~\"$job\"}",
|
||||
"legendFormat": "{{job}} llama.cpp prefill tok/s"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
Reference in new issue
Block a user