Frank
701ab14f60
Rewrites the [[Observability]] umbrella page and the five component pages for what is actually deployed: rootless Quadlet units instead of compose services, the loopback-only publish posture (node-exporter's LAN binding explained), the loki healthcheck living in its image because the image has no shell, and the Home=/root trap that killed the first grafana start. Adds the note that matters most to a reader: the stack is installed, healthy and UNCONFIGURED. Grafana reports zero datasources and zero dashboards with an empty provisioning directory; Prometheus has no rule files and no Alertmanager; Loki is configured but has no reader; node-exporter runs its default collectors. Alloy is the only configured piece. Metrics and logs are collected and retained, and nothing is visualized or alerted on. AGENTS.md's boot-persistence gap note is replaced by the resolution.
73 lines
3.3 KiB
Markdown
73 lines
3.3 KiB
Markdown
# Grafana
|
|
|
|
Grafana visualizes the metrics that Prometheus collects from every service. It runs on Dankclaw and is reachable at `grafana.franksdank.cloud` (via Caddy, behind the Anubis challenge + embedded-outpost gate; login happens inside Grafana via the Authentik OAuth2 button).
|
|
|
|
**Host:** Dankclaw — **Hostname:** `grafana.franksdank.cloud` — **Port:** 3002 (internal) — **Docs:** https://grafana.com/docs
|
|
|
|
## Facts
|
|
|
|
| Item | Value |
|
|
|------|-------|
|
|
| Image | `docker.io/grafana/grafana:latest` |
|
|
| Published | `127.0.0.1:3002` → 3000, loopback only (see [[Observability]] → Exposure) |
|
|
| Unit | `dankclaw/observability/quadlet/grafana.container` → `grafana.service` |
|
|
| Data | `/opt/grafana/data` — `grafana.db`: accounts, sessions, anything created in the UI |
|
|
| Provisioning mount | `/opt/grafana/provisioning` (read-only) — **empty as of 2026-09-11** |
|
|
| Datasource | **none configured** — see the state note in [[Observability]] |
|
|
|
|
## Configuration
|
|
|
|
Everything comes from the Quadlet unit. The Authentik OAuth2 client secret is
|
|
the only credential, and systemd expands it from a per-service env file:
|
|
|
|
```
|
|
[Service]
|
|
EnvironmentFile=/home/dank/Containers/grafana/env # systemd-side only
|
|
[Container]
|
|
Environment=GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET=${GRAFANA_OAUTH_CLIENT_SECRET}
|
|
```
|
|
|
|
`seed_grafana_env` in `scripts/deploy-remote.sh` seeds that file from
|
|
`dankclaw/.env` with a single-variable filter and fails the deploy when the
|
|
value is missing. The container receives the one expanded value, never the file.
|
|
|
|
## Datasource
|
|
|
|
None is configured. Grafana cannot query anything yet; pointing it at
|
|
`http://prometheus:9090` and `http://loki:3100` (by container name, on the
|
|
`dankclaw_default` network) is outstanding work — see the state note in
|
|
[[Observability]].
|
|
|
|
## Healthcheck
|
|
|
|
Grafana responds on `/api/health` with 200 and `"database":"ok"`.
|
|
|
|
## Gotchas
|
|
|
|
- Change the default admin credentials at first boot (env or UI).
|
|
- Dashboards provisioned as YAML survive container recreation; UI-created ones live in the data volume — back it up.
|
|
- **Auth = OAuth2 relying-party, not auth_proxy.** The embedded outpost carries
|
|
no provider token (`token=None`), so no identity header ever reaches Grafana;
|
|
RP mode is topology-independent. The full verified contract (provider pk 9,
|
|
client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths,
|
|
Basic client auth, form-encoded token body, and the fork's userinfo-claims
|
|
bug that leaves identity at the `sub` hash only) is in
|
|
[[Authentik]] → "OAuth2 relying-party quirks". Consequences for Grafana:
|
|
users auto-upsert as Viewer with `login = <sub hex>`; no email/groups/role
|
|
claims reach Grafana until the evaluator bug is fixed — do not write role
|
|
mappings that depend on those claims. The local admin account stays the only
|
|
Admin.
|
|
- **Never point this unit's `EnvironmentFile` at the repo-root `.env`.** That
|
|
file is a gluetun env file and it carries `HOME=/root`; systemd imports it
|
|
for the service, podman then looks for its configuration under `/root`,
|
|
cannot read it, and the unit dies at start with
|
|
`stat /root/.config: permission denied`. The unit reads
|
|
`/home/dank/Containers/grafana/env`, which the deployer seeds with exactly
|
|
one variable.
|
|
|
|
## See Also
|
|
|
|
- [[Dankclaw Stack]]
|
|
- [[Prometheus]] — the datasource
|
|
- [[Observability]] — the whole stack
|