docs(observability): record the quadlet cutover, the exposure posture, and the unconfigured state

Rewrites the [[Observability]] umbrella page and the five component pages for
what is actually deployed: rootless Quadlet units instead of compose services,
the loopback-only publish posture (node-exporter's LAN binding explained), the
loki healthcheck living in its image because the image has no shell, and the
Home=/root trap that killed the first grafana start.

Adds the note that matters most to a reader: the stack is installed, healthy
and UNCONFIGURED. Grafana reports zero datasources and zero dashboards with an
empty provisioning directory; Prometheus has no rule files and no Alertmanager;
Loki is configured but has no reader; node-exporter runs its default
collectors. Alloy is the only configured piece. Metrics and logs are collected
and retained, and nothing is visualized or alerted on.

AGENTS.md's boot-persistence gap note is replaced by the resolution.

serverAppConfigs-docs/Infrastructure/Grafana.md

@@ -9,26 +9,34 @@ Grafana visualizes the metrics that Prometheus collects from every service. It r
 | Item | Value |
 |------|-------|
 | Image | `docker.io/grafana/grafana:latest` |
-| Port | 3002 (host) → 3000 (container) |
-| Data volume | dashboards, users, datasource config |
-| Datasource | Prometheus (on the same host) |
-| Provisioning | dashboards in `/etc/grafana/provisioning/dashboards` |
+| Published | `127.0.0.1:3002` → 3000, loopback only (see [[Observability]] → Exposure) |
+| Unit | `dankclaw/observability/quadlet/grafana.container` → `grafana.service` |
+| Data | `/opt/grafana/data` — `grafana.db`: accounts, sessions, anything created in the UI |
+| Provisioning mount | `/opt/grafana/provisioning` (read-only) — **empty as of 2026-09-11** |
+| Datasource | **none configured** — see the state note in [[Observability]] |
 
 ## Configuration
 
-```yaml
-environment:
-  GF_SECURITY_ADMIN_USER: "${GRAFANA_ADMIN_USER}"
-  GF_SECURITY_ADMIN_PASSWORD: "${GRAFANA_ADMIN_PASSWORD}"
-  GF_SERVER_ROOT_URL: "https://grafana.franksdank.cloud"
-volumes:
-  - /opt/grafana/data:/var/lib/grafana
-  - /opt/grafana/provisioning:/etc/grafana/provisioning:ro
+Everything comes from the Quadlet unit. The Authentik OAuth2 client secret is
+the only credential, and systemd expands it from a per-service env file:
+
 ```
+[Service]
+EnvironmentFile=/home/dank/Containers/grafana/env      # systemd-side only
+[Container]
+Environment=GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET=${GRAFANA_OAUTH_CLIENT_SECRET}
+```
+
+`seed_grafana_env` in `scripts/deploy-remote.sh` seeds that file from
+`dankclaw/.env` with a single-variable filter and fails the deploy when the
+value is missing. The container receives the one expanded value, never the file.
 
 ## Datasource
 
-Point Grafana at the Prometheus service by name (`http://prometheus:9090`) on the shared podman network.
+None is configured. Grafana cannot query anything yet; pointing it at
+`http://prometheus:9090` and `http://loki:3100` (by container name, on the
+`dankclaw_default` network) is outstanding work — see the state note in
+[[Observability]].
 
 ## Healthcheck
 
@@ -49,6 +57,13 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`.
   claims reach Grafana until the evaluator bug is fixed — do not write role
   mappings that depend on those claims. The local admin account stays the only
   Admin.
+- **Never point this unit's `EnvironmentFile` at the repo-root `.env`.** That
+  file is a gluetun env file and it carries `HOME=/root`; systemd imports it
+  for the service, podman then looks for its configuration under `/root`,
+  cannot read it, and the unit dies at start with
+  `stat /root/.config: permission denied`. The unit reads
+  `/home/dank/Containers/grafana/env`, which the deployer seeds with exactly
+  one variable.
 
 ## See Also