fix(deploy): grafana copier provisions EVERY dashboard json, not a hand-listed subset

copy_grafana_provisioning named llm-fleet.json alone, so a new board in
shared/grafana/provisioning/dashboards/ silently never reached
/opt/grafana/provisioning. Measured 2026-10-21: the containers board was
merged, seam-tested, and documented as working since 2026-10-20, yet was
NEVER provisioned — its panel fixes were undeployable; the other four
boards existed on disk only because someone hand-copied them once (and
host-system.json there had gone stale against the repo). The copier now
globs the directory, and GRAFANA_PROV_ROOT lets the new bats case run
the copier for real into a sandbox (red against the old hand-listed
body, green against the glob). Vault: the copier law + the containers
incident on Observability Provisioning, the corrected dashboards claim
on Observability.

Note: a mktemp sandbox + trap RETURN races bats' trap machinery (the
dir vanishes mid-function); BATS_TEST_TMPDIR is the safe idiom.
This commit is contained in:
Frank committed 2026-10-21 09:40:57 -07:00
1 parent 127621bdd7
commit c41384e03d
4 files changed
+69 -23

No files matched your search

@@ -109,12 +109,15 @@ verify it through the APIs.
* **Delivery: working.** Alertmanager's single `signal` receiver webhooks the
relay, and `journalctl --user -u signal-relay` records successful
`sent to +12062476336:` lines for every rule in the set.
* **Dashboards: working.** Provisioned datasources — Prometheus (uid
`prometheus`) and Loki (uid `loki`) — and the `llm-fleet`, `host-system`,
`containers`, `observability` and `logs-overview` dashboards, all
installed from `shared/grafana/provisioning/`. The Loki datasource is how
you read logs: Loki has no web UI of its own, so Grafana Explore is the
human surface (the guarded `loki.franksdank.cloud` vhost serves only the API).
* **Dashboards: working (after the 2026-10-21 copier fix).** Provisioned
datasources — Prometheus (uid `prometheus`) and Loki (uid `loki`) — and the
`llm-fleet`, `host-system`, `containers`, `observability` and
`logs-overview` boards. Until 2026-10-21 the copier carried only
`llm-fleet.json` by name and the `containers` board had never been
provisioned; the deployer now globs the directory. The Loki datasource is
how you read logs: Loki has no web UI of its own, so Grafana Explore is the
human surface (the guarded `loki.franksdank.cloud` vhost serves only the
API). Verify method and the copier law: [[Observability Provisioning]].
* **Per-container metrics: the `containers` dashboard (added 2026-10-20; live
after `deploy.sh <host> --service cadvisor`).** cadvisor is the only way to see
per-container CPU/memory here, and on this rootless-Podman host it works on one