alerting: Alertmanager + Signal relay deliver firing rules to Frank's phone
- dankclaw/observability/quadlet/alertmanager.container: loopback :9093, config from shared/prometheus/alertmanager.yml, family conventions - shared/prometheus/alertmanager.yml: single signal receiver, group_by alertname, 30s/5m/4h intervals, webhook to host.containers.internal:18093 - scripts/signal-relay.py + .service: host-side user unit, loopback :18093, 15-min dedupe, sends via signal-cli account 318311 - prometheus.yml: alerting block to alertmanager:9093; stale comments fixed - llm-fleet.yml: delivery header; LlmSpecDecodeOff annotation now drafter-agnostic - deploy.sh/deploy-remote.sh/deploy-plan.py: alertmanager service, config copy, relay install, planner routing for config + relay files - homepage: alertmanager status card (loopback probe) - tests: alertmanager in UNITS/loopback/health/PAIRS/planner + five new delivery-path seams (webhook<->relay contract, prometheus target, relay unit, signal-cli account, planner routing) - docs: Alertmanager.md component page, Observability.md, Prometheus.md, index.md, AGENTS.md delivery line
This commit is contained in:
1 parent
17450dd172
commit
590853c133
17 files changed
+513
-51
No files matched your search
@@ -122,14 +122,17 @@ This rule exists because reboots found gaps twice (2026-08-26: Caddy as a
|
||||
bare compose container never returned; 2026-09-11: five observability
|
||||
services had no units). The observability services now have units under
|
||||
`dankclaw/observability/quadlet/` and their compose definitions were removed —
|
||||
one owner per container name. The stack is **configured as of 2026-09-16**: a
|
||||
Grafana datasource and the `llm-fleet` dashboard are provisioned from
|
||||
`shared/grafana/provisioning/`, and `shared/prometheus/rules/llm-fleet.yml`
|
||||
loads 5 alert rules + 1 recording rule. Nothing is *delivered* — there is no
|
||||
Alertmanager, so a firing alert reaches nobody (an operator decision, not an
|
||||
oversight). Five Prometheus targets read down for structural reasons
|
||||
(HTML-scraped "probes", Gitea metrics off); the remedy is a blackbox exporter.
|
||||
See `serverAppConfigs-docs/Infrastructure/Observability.md`.
|
||||
one owner per container name. The stack is **configured and delivering as of
|
||||
2026-09-17**: a Grafana datasource and the `llm-fleet` dashboard are
|
||||
provisioned from `shared/grafana/provisioning/`, `shared/prometheus/rules/
|
||||
llm-fleet.yml` loads 5 alert rules + 1 recording rule, and firing rules are
|
||||
delivered: Prometheus → Alertmanager (`dankclaw/observability/quadlet/
|
||||
alertmanager.container`, loopback :9093) → host-side Signal relay
|
||||
(`scripts/signal-relay.py`, user unit, loopback :18093, reached from the
|
||||
container as `host.containers.internal`) → signal-cli account 318311. Five
|
||||
Prometheus targets read down for structural reasons (HTML-scraped "probes",
|
||||
Gitea metrics off); the remedy is a blackbox exporter. See
|
||||
`serverAppConfigs-docs/Infrastructure/Observability.md`.
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
Reference in new issue
Block a user