docs(observability): record the quadlet cutover, the exposure posture, and the unconfigured state
Rewrites the [[Observability]] umbrella page and the five component pages for what is actually deployed: rootless Quadlet units instead of compose services, the loopback-only publish posture (node-exporter's LAN binding explained), the loki healthcheck living in its image because the image has no shell, and the Home=/root trap that killed the first grafana start. Adds the note that matters most to a reader: the stack is installed, healthy and UNCONFIGURED. Grafana reports zero datasources and zero dashboards with an empty provisioning directory; Prometheus has no rule files and no Alertmanager; Loki is configured but has no reader; node-exporter runs its default collectors. Alloy is the only configured piece. Metrics and logs are collected and retained, and nothing is visualized or alerted on. AGENTS.md's boot-persistence gap note is replaced by the resolution.
This commit is contained in:
1 parent
25c93984ad
commit
701ab14f60
8 files changed
+220
-147
No files matched your search
@@ -200,15 +200,18 @@ Pi-hole unit existed but crash-looped on a sysctl conflict (a later-sorted
|
||||
drop-in reset `net.ipv4.ip_unprivileged_port_start` to 80), taking LAN DNS
|
||||
down with it.
|
||||
|
||||
**Known gap (2026-09-11, found while rebooting Dankclaw for the `amd_iommu=off`
|
||||
change):** five Dankclaw services still run only from `dankclaw/compose.yaml`
|
||||
with no Quadlet unit — `prometheus`, `grafana`, `loki`, `alloy` and
|
||||
`node-exporter` (all under `~/.config/containers/systemd/` absent, but present
|
||||
in the compose file). They did not come back after that reboot and were
|
||||
restored with `./scripts/deploy.sh dankclaw --service <name>`; they will not
|
||||
survive the next one either. Giving them units is tracked work, not a solved
|
||||
problem — treat every reboot as expected to take the observability stack down
|
||||
until it is done.
|
||||
**Observability gap (found 2026-09-11, fixed the same day):** rebooting
|
||||
Dankclaw for the `amd_iommu=off` change exposed five services that ran only
|
||||
from `dankclaw/compose.yaml` with no Quadlet unit — `prometheus`, `grafana`,
|
||||
`loki`, `alloy` and `node-exporter`. They did not come back and had to be
|
||||
restored by hand. They are now rootless Quadlet units under
|
||||
`dankclaw/observability/quadlet/` (`./scripts/deploy.sh dankclaw --service
|
||||
<name>`), and their compose definitions were removed so only one owner exists
|
||||
per container name. The three HTTP UIs publish on loopback only; off-host
|
||||
access is the guarded vhost.
|
||||
Note that the stack is RUNNING BUT UNCONFIGURED: Grafana has no datasources
|
||||
and no dashboards, and Prometheus has no rules and no Alertmanager. See
|
||||
`serverAppConfigs-docs/Infrastructure/Observability.md`.
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
Reference in new issue
Block a user