docs(observability): post-blackbox state, copier law, label-promotion why

Prometheus/Observability/Provisioning pages: every job up after the
blackbox swap, the copier list learns copy_blackbox_config and the
alloy validate gate, the config table gains alloy + blackbox rows, and
the logs bullet states the stream-label law instead of the retired
'awaiting deploy' note.

serverAppConfigs-docs/Infrastructure/Observability.md

@@ -95,13 +95,14 @@ health endpoint (`/prometheus`: `/-/healthy`, `/alertmanager`: `/-/healthy`
 [[Observability Provisioning]] for what is provisioned as code and how to
 verify it through the APIs.
 
-## Current state (2026-10-20)
+## Current state (2026-10-21)
 
 * **Metrics: working.** `curl -s http://127.0.0.1:9090/api/v1/targets` reports
-  eight up targets — the three LLM endpoints (frank-archon on `vllm:*`,
-  frank-dragoon and frank-zealot on `llamacpp:*`), grafana, loki, prometheus
-  and both node exporters. The four downs are structural (the `comfyui` and
-  `services` HTML probes, and Gitea's disabled metrics endpoint), not outages.
+  every job up — the three LLM endpoints (frank-archon on `vllm:*`,
+  frank-dragoon and frank-zealot on `llamacpp:*`), grafana, loki, prometheus,
+  both node exporters, cadvisor, gitea (metrics enabled in its quadlet) and
+  the blackbox-probed `services` (SearXNG, OpenWebUI, ComfyUI). The structural
+  downs are retired — see [[Prometheus]] → Targets.
   Two rule groups in `shared/prometheus/rules/llm-fleet.yml`: `llm-fleet`
   (2 recording + 4 alert, `llamacpp:*`) and `vllm-fleet` (1 recording + 4
   alert, `vllm:*` — archon queueing, prefix collapse, preemption, p95
@@ -142,26 +143,26 @@ verify it through the APIs.
   `power1_average`, so they carry no `gpu_power_watts` line at all (only NVIDIA and
   the Dankclaw 8060S iGPU report power); and it binds loopback + the host LAN, like
   node-exporter, never the wildcard. The dashboard graphs util/VRAM/VRAM-%/temp/power.
-* **Logs: fixed in code, awaiting deploy.** The pipeline shipped nothing because
-  a rootless Alloy cannot read the `0640` journal files, and the container-log
-  globs point at a root-storage layout that does not exist here. The fix is built
-  and tested in-repo (2026-10-20): Alloy is now a **root system service on
-  `Network=host`**, reading `system.journal` + `user-*.journal` and pushing to
-  Loki's `127.0.0.1:3100` loopback publish — a throwaway root probe read 31,825
-  lines and pushed 26,966 with zero drops. The **live** instance is still the old
-  rootless unit until `deploy.sh dankclaw --service alloy` runs. Detail in
-  [[Alloy]] → Why root; Loki's store state in [[Loki]] → Contents.
-* **Not configured:** no blackbox exporter (the structural downs have no real
-  uptime signal) and no Alertmanager vhost. The log pipeline's fix is in code but
-  not yet deployed (see *Logs* above).
+* **Logs: flowing; container labels land with the 2026-10-21 deploy.** The
+  pipeline once shipped nothing (rootless Alloy cannot read the `0640` journal;
+  the container-log globs pointed at a root-storage layout that does not exist
+  here) — fixed 2026-10-20 by making Alloy a **root system service on
+  `Network=host`**. Journal lines now reach Loki (measured ~6.5k lines/5min),
+  but the stream carried only `job`/`service_name`, so every per-container
+  view was structurally empty: Loki metrics queries group only by INDEXED
+  stream labels. Alloy now promotes podman's `CONTAINER_NAME` to a `container`
+  label (plus a `host` label) at ingestion — `loki.process` in
+  `shared/alloy/config.alloy`; detail in [[Alloy]] → Why root, and the
+  purpose-test law in [[Observability Provisioning]].
 
 ## Citations
 
-- `dankclaw/observability/quadlet/{prometheus,alertmanager,grafana,loki,alloy,node-exporter,cadvisor}.container`
+- `dankclaw/observability/quadlet/{prometheus,alertmanager,grafana,loki,alloy,node-exporter,cadvisor,blackbox-exporter}.container`
 - `scripts/deploy-remote.sh:219-300` (copiers + relay), `:521-536` (health waits), `:477-486` (grafana env seed)
 - `scripts/signal-relay.py:26-49`, `scripts/signal-relay.service`
 - `scripts/gpu-exporter.py`, `scripts/gpu-exporter.service` (per-GPU host exporter)
 - `shared/prometheus/prometheus.yml`, `shared/prometheus/alertmanager.yml`, `shared/prometheus/rules/llm-fleet.yml`
 - `shared/grafana/provisioning/`, `shared/loki/loki-config.yaml`, `shared/alloy/config.alloy`
+- `shared/blackbox/blackbox.yml` (prober modules + loopback bind)
 - `tests/observability_quadlet.bats` (boot persistence, no wildcard publish, probe addresses, delivery contract)
 - Host evidence 2026-10-18: `ss -ltnp`, `podman ps`, `systemctl --user is-enabled|is-active`, `curl` on `127.0.0.1:9090/api/v1/{targets,rules,alerts}`, `127.0.0.1:3100/loki/api/v1/labels`, `127.0.0.1:12345/metrics`, `127.0.0.1:3002/api/health`, `192.168.0.143:18093/health`, `journalctl --user -u signal-relay`