docs(observability): correct current state and log-pipeline root cause

AGENTS.md: the rule set is now two groups (llm-fleet + vllm-fleet) and a
Loki datasource is provisioned beside Prometheus.
Observability.md: current-state census refreshed (three LLM targets up,
two rule groups, Loki datasource); logs still carry no data.
Alloy.md: replace the 'wrong glob' theory with the proven cause - a
rootless container cannot read the 0640 systemd-journal files (a rootless
busybox read returns DENIED), and rootless quadlet logs land in the user
journal the system-only source never reads. Closing it is a
permission/architecture decision, not a config glob.

serverAppConfigs-docs/Infrastructure/Observability.md

@@ -3,7 +3,7 @@ type: reference
 title: "Observability on Dankclaw"
 description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured."
 tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
-timestamp: 2026-09-11
+timestamp: 2026-10-20
 ---
 
 # Observability on Dankclaw
@@ -83,27 +83,36 @@ health endpoint (`/prometheus`: `/-/healthy`, `/alertmanager`: `/-/healthy`
 [[Observability Provisioning]] for what is provisioned as code and how to
 verify it through the APIs.
 
-## Current state (2026-10-18)
+## Current state (2026-10-20)
 
 * **Metrics: working.** `curl -s http://127.0.0.1:9090/api/v1/targets` reports
-  eight up targets; the four downs are structural (the `comfyui` and `services`
-  HTML probes, and Gitea's disabled metrics endpoint), not outages. Rule set:
-  4 alert rules + 2 recording rules (`/api/v1/rules`, matching
-  `shared/prometheus/rules/llm-fleet.yml`).
+  eight up targets — the three LLM endpoints (frank-archon on `vllm:*`,
+  frank-dragoon and frank-zealot on `llamacpp:*`), grafana, loki, prometheus
+  and both node exporters. The four downs are structural (the `comfyui` and
+  `services` HTML probes, and Gitea's disabled metrics endpoint), not outages.
+  Two rule groups in `shared/prometheus/rules/llm-fleet.yml`: `llm-fleet`
+  (2 recording + 4 alert, `llamacpp:*`) and `vllm-fleet` (1 recording + 4
+  alert, `vllm:*` — archon queueing, prefix collapse, preemption, p95
+  time-to-first-token), each keyed only to the namespace its runtime exports.
 * **Delivery: working.** Alertmanager's single `signal` receiver webhooks the
   relay, and `journalctl --user -u signal-relay` records successful
   `sent to +12062476336:` lines for every rule in the set.
-* **Dashboards: working.** One provisioned Prometheus datasource (uid
-  `prometheus`) and the `llm-fleet` dashboard, both installed from
-  `shared/grafana/provisioning/`.
-* **Logs: configured, empty.** Alloy ships zero lines (journal counter 0, both
-  file globs refused) and Loki therefore holds zero streams — `/opt/loki/data`
-  is a few hundred KB of WAL/compactor bookkeeping with **no chunk files**
-  (8 files total by `find`, 2026-10-18) and `GET /loki/api/v1/labels` returns
-  no label names.
-  See [[Alloy]] → Delivery state and [[Loki]] → Contents.
-* **Not configured:** no Loki datasource in Grafana, no blackbox exporter (the
-  structural downs have no real uptime signal), no Alertmanager vhost.
+* **Dashboards: working.** Provisioned datasources — Prometheus (uid
+  `prometheus`) and Loki (uid `loki`) — and the `llm-fleet` dashboard, all
+  installed from `shared/grafana/provisioning/`. The Loki datasource is how
+  you read logs: Loki has no web UI of its own, so Grafana Explore is the
+  human surface (the guarded `loki.franksdank.cloud` vhost serves only the API).
+* **Logs: configured, empty.** Alloy ships zero lines and Loki holds zero
+  streams (`GET /loki/api/v1/labels` returns no label names; no chunk files on
+  disk). The cause is source-side access, now proven, not a config bug: a
+  rootless container cannot read the journal files (`system.journal` is
+  `root:systemd-journal 0640`/`adm` and a rootless read of it returns DENIED),
+  and the container-log globs point at a root-storage path that does not exist
+  under rootless. Closing it is a permission/architecture decision, not an
+  edit — see [[Alloy]] → Delivery state and [[Loki]] → Contents.
+* **Not configured:** no blackbox exporter (the structural downs have no real
+  uptime signal) and no Alertmanager vhost. The log pipeline is provisioned
+  but carries no data (see *Logs* above).
 
 ## Citations