docs: record the alerting rules and what they still cannot do
Both pages claimed there was no alerting; there are six rules now, so they say which rules exist, that promtool and /api/v1/rules both confirm them, and — in the same breath — that no rule reaches a person, because this stack has no Alertmanager and no contact point. The dashboard's ALERTS panel is named as the only place an alert becomes visible without opening Prometheus.
This commit is contained in:
1 parent
af05c890fd
commit
823814920b
2 files changed
+13
-4
No files matched your search
@@ -418,8 +418,16 @@ scraped — Prometheus has `frank-medium` and `frank-large` jobs
|
||||
(queue depth), `rate(vllm:prefix_cache_hits_total[5m]) /
|
||||
rate(vllm:prefix_cache_queries_total[5m])` (whether pinned conversations are
|
||||
served warm), and the request-latency histograms. See
|
||||
[[Observability on Dankclaw]]. There is still **no alerting** in this stack, so
|
||||
a saturated pool will not page anyone — it shows up on a graph after the fact.
|
||||
[[Observability on Dankclaw]], and one Grafana dashboard reads them
|
||||
(**LLM fleet — capacity and reuse**).
|
||||
|
||||
Five rules watch this service (`shared/prometheus/rules/llm-fleet.yml`): pool
|
||||
saturation above 90 %, requests queueing for 10 minutes, prefix reuse collapsing
|
||||
below 20 % with traffic, the scrape target down, and p95 latency above 60 s —
|
||||
each threshold tied to a measurement on this page rather than to a guess. They
|
||||
evaluate and show up on the dashboard's **Alerts firing** panel, but **nothing
|
||||
is notified**: this stack has no Alertmanager, so an alert reaches a person only
|
||||
if that person is looking. That is the honest state of "alerting" here.
|
||||
|
||||
## Citations
|
||||
|
||||
|
||||
Reference in new issue
Block a user