alerting: Alertmanager + Signal relay deliver firing rules to Frank's phone

- dankclaw/observability/quadlet/alertmanager.container: loopback :9093,
  config from shared/prometheus/alertmanager.yml, family conventions
- shared/prometheus/alertmanager.yml: single signal receiver, group_by
  alertname, 30s/5m/4h intervals, webhook to host.containers.internal:18093
- scripts/signal-relay.py + .service: host-side user unit, loopback :18093,
  15-min dedupe, sends via signal-cli account 318311
- prometheus.yml: alerting block to alertmanager:9093; stale comments fixed
- llm-fleet.yml: delivery header; LlmSpecDecodeOff annotation now drafter-agnostic
- deploy.sh/deploy-remote.sh/deploy-plan.py: alertmanager service, config
  copy, relay install, planner routing for config + relay files
- homepage: alertmanager status card (loopback probe)
- tests: alertmanager in UNITS/loopback/health/PAIRS/planner + five new
  delivery-path seams (webhook<->relay contract, prometheus target, relay
  unit, signal-cli account, planner routing)
- docs: Alertmanager.md component page, Observability.md, Prometheus.md,
  index.md, AGENTS.md delivery line

serverAppConfigs-docs/Infrastructure/Alertmanager.md

@@ -0,0 +1,50 @@
+# Alertmanager
+
+Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to the host-side Signal relay — which sends it through signal-cli to Frank's phone. It is the first delivery path in the observability stack (2026-09-17); before it, a firing alert was visible in Prometheus and on the Grafana dashboard's ALERTS panel and reached nobody.
+
+**Host:** Dankclaw — **Port:** 9093 (loopback) — **Docs:** https://prometheus.io/docs/alerting/latest/alertmanager/
+
+## Facts
+
+| Item | Value |
+|------|-------|
+| Image | `docker.io/prom/alertmanager:latest` |
+| Published | `127.0.0.1:9093` → 9093, loopback only (see [[Observability]] → Exposure) |
+| Unit | `dankclaw/observability/quadlet/alertmanager.container` → `alertmanager.service` |
+| Config | `/opt/alertmanager/config/alertmanager.yml`, installed from `shared/prometheus/alertmanager.yml` on every deploy |
+| Data | `/opt/alertmanager/data` (silences, notification log) |
+| Health | `wget --spider -q http://localhost:9093/-/healthy` |
+| Receiver | one `signal` receiver → webhook `http://host.containers.internal:18093/alertmanager` |
+| Grouping | `group_by: [alertname]`, `group_wait` 30 s, `group_interval` 5 m, `repeat_interval` 4 h |
+
+## Why the webhook address is `host.containers.internal`
+
+The Signal relay is a host-side process on the host's **loopback** (127.0.0.1:18093). From a bridge container, loopback is the container's own, and netavark DNATs *published* ports only — so the LAN IP and the podman gateway both refuse an unpublished host bind. `host.containers.internal` (169.254.1.2) is the one address that reaches a host-local listener from a bridge container. The relay listens on loopback only, so the unauthenticated webhook endpoint stays off every host interface.
+
+## The Signal relay
+
+`scripts/signal-relay.py` is a systemd **user** unit (`scripts/signal-relay.service`), not a Quadlet: it runs `python3` from the checkout, so a deploy reaches it with no copy step, and a hand-written user unit can be `enable`d (generated quadlet units cannot on this host). Linger is on for `dank`, so `WantedBy=default.target` is the whole boot story.
+
+- POST `/alertmanager` — the Alertmanager webhook handler; formats the group and sends it.
+- GET `/health` — liveness for the deployer's health wait.
+- Dedupes per group for 15 minutes, so a stuck alert cannot out-repeat the 4 h `repeat_interval` during a relay hiccup.
+- Sends through `/usr/local/bin/signal-cli -a 318311 send +12062476336` — Frank's existing account; the credential is the account state under `~/.local/share/signal-cli`, and the account/phone are numbers, not secrets.
+
+## Deployment
+
+```bash
+./scripts/deploy.sh dankclaw --service alertmanager
+```
+
+`deploy_observability` installs the quadlet unit, `copy_alertmanager_config` installs the config, and `install_signal_relay` copies the user unit, enables and restarts it, then waits on `127.0.0.1:18093/health` before the container's own `/-/healthy` wait. The planner selects `alertmanager` for its quadlet file, `shared/prometheus/alertmanager.yml`, and the two relay files; `prometheus` ships the `alerting:` block that points at `alertmanager:9093`.
+
+## Gotchas
+
+- The webhook URL and the relay's listen port are a **cross-file contract**: the relay moves ports and the webhook string does not move with it, and every alert is then dropped with no error anywhere. `tests/observability_quadlet.bats` pins both halves.
+- The container address is `alertmanager:9093`, never the published port: Prometheus is in a container and the port is loopback-only.
+- signal-cli account state is on the host, not in the container — a wiped `~/.local/share/signal-cli` silences delivery while every status page stays green.
+
+## See Also
+
+- [[Prometheus]] — the rule evaluator that ships the alerts
+- [[Observability]] — the full stack