observability: LLM alerting rules, evaluated but not yet delivered

The scraped metrics have no rules over them, so every graph was read after the
fact by a human. This adds shared/prometheus/rules/llm-fleet.yml, loaded via
rule_files and installed by the same copier that ships prometheus.yml (a rule
file nothing loads evaluates nothing, and Prometheus reports no error for it):

  LlmKvPoolSaturated      kv pool > 90% for 10m — the 122,636-token wall
  LlmRequestsQueueing     waiting > 0 for 10m — MAXSEQS=4 reached
  LlmPrefixCacheCollapse  reuse < 20% for 15m with traffic — the thrash signature
  LlmEndpointDown         scrape target down 2m — both replicas gone
  LlmLatencyHigh          p95 end-to-end > 60s for 10m — what an operator feels
  job:llm_prefix_cache_reuse:rate5m — recording rule shared by the alert and the
                          dashboard, so the two cannot drift

Every threshold is tied to something measured on this fleet, not to a guess,
and the file says plainly at the top that **nothing is notified**: there is no
Alertmanager here, so a firing alert reaches Prometheus' /alerts page, the
ALERTS metric and the dashboard panel added alongside it — and nowhere else.
Choosing how Frank actually gets paged is his call, not a missing config line.

Seam covers the silent failure mode: rules exist, the config references the
directory, the deployer installs it, and the dashboard graphs ALERTS.

tests/observability_quadlet.bats

@@ -324,3 +324,31 @@ PAIRS
     [ "$status" -eq 0 ]
     [ "$output" = "grafana" ]
 }
+
+@test "LLM alerting rules exist and are referenced by the config that ships them" {
+    local rules="$REPO_ROOT/shared/prometheus/rules/llm-fleet.yml"
+    local cfg="$REPO_ROOT/shared/prometheus/prometheus.yml"
+    [ -f "$rules" ]
+    python3 -c "
+import yaml
+d = yaml.safe_load(open('$rules'))
+names = [r.get('alert') or r.get('record') for g in d['groups'] for r in g['rules']]
+assert 'record' or True
+for want in ('LlmKvPoolSaturated', 'LlmRequestsQueueing', 'LlmPrefixCacheCollapse',
+             'LlmEndpointDown', 'LlmLatencyHigh'):
+    assert want in names, want
+print('rules ok:', len(names))
+"
+    # A rule file that nothing loads evaluates nothing, and Prometheus reports
+    # no error for a missing rule_files entry — the failure is invisible.
+    grep -q '/etc/prometheus/rules/\*.yml' "$cfg"
+    # ...and the deployer must install the directory the config points at.
+    grep -q 'cp shared/prometheus/rules/llm-fleet.yml' "$REPO_ROOT/scripts/deploy-remote.sh"
+    # The dashboard surfaces firing alerts, since nothing else will.
+    python3 -c "
+import json
+d = json.load(open('$REPO_ROOT/shared/grafana/provisioning/dashboards/llm-fleet.json'))
+exprs = [t['expr'] for p in d['panels'] for t in p.get('targets', [])]
+assert any('ALERTS' in e for e in exprs), exprs
+"
+}