dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(observability): record the quadlet cutover, the exposure posture, and the unconfigured state

Browse Source 
Rewrites the [[Observability]] umbrella page and the five component pages for
what is actually deployed: rootless Quadlet units instead of compose services,
the loopback-only publish posture (node-exporter's LAN binding explained), the
loki healthcheck living in its image because the image has no shell, and the
Home=/root trap that killed the first grafana start.

Adds the note that matters most to a reader: the stack is installed, healthy
and UNCONFIGURED. Grafana reports zero datasources and zero dashboards with an
empty provisioning directory; Prometheus has no rule files and no Alertmanager;
Loki is configured but has no reader; node-exporter runs its default
collectors. Alloy is the only configured piece. Metrics and logs are collected
and retained, and nothing is visualized or alerted on.

AGENTS.md's boot-persistence gap note is replaced by the resolution.
This commit is contained in:
Frank committed 2026-09-11 16:10:27 -07:00
1 parent 25c93984ad
commit 701ab14f60
8 files changed
+220 -147
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

serverAppConfigs-docs/Infrastructure/Alloy.md
+24 -17
View File
@@ -1,47 +1,54 @@
# Alloy
Alloy is Grafana's log and telemetry shipper. It replaces Promtail (EOL 2026-03-02). One Alloy instance runs per host and forwards container and journal logs to Loki, and optionally metrics to Prometheus.
Alloy is Grafana's log and telemetry shipper. It replaces Promtail (EOL 2026-03-02). It forwards container and journal logs to Loki.
**Hosts:** Dankclaw + Dankserver (one per host) — **Docs:** https://grafana.com/docs/alloy/latest
**Host:** Dankclaw (one instance per host) — **Port:** 12345 (loopback) — **Docs:** https://grafana.com/docs/alloy/latest
## Facts (from research)
## Facts
| Item | Value |
|------|-------|
| Image | `franksdank/alloy:latest` (local build with a static busybox healthcheck) |
| Port | none published (push-only) |
| Config | River syntax, `config.alloy` |
| Image | `franksdank/alloy:latest` (local build: upstream + a static busybox for the healthcheck) |
| Published | `127.0.0.1:12345` → 12345 (its own UI/health port) |
| Unit | `dankclaw/observability/quadlet/alloy.container` → `alloy.service` |
| Config | `shared/alloy/config.alloy`, installed to `/opt/alloy/config/` |
| Inputs | host journald socket, podman container logs, `/var/log` |
| Output | `http://loki:3100/loki/api/v1/push` |
| Successor to | Promtail (EOL 2026-03-02) |
| Inputs | journald socket, container log files |
| Output | Loki push endpoint |
## Why Alloy and not Promtail
Promtail reached end-of-life on 2026-03-02. Grafana Labs directs new installs to Alloy. This stack uses Alloy from day one.
Promtail reached end-of-life on 2026-03-02 and Grafana Labs directs new installs to Alloy. This stack uses Alloy from day one.
## Configuration
River config at `shared/alloy/config.alloy`:
`shared/alloy/config.alloy` is the source of truth:
- Read journald logs from the host's `/run/systemd/journal/socket`.
- Read container logs from `/var/lib/containers/...` (podman) or the docker path.
- Forward to `http://loki:3100/loki/api/v1/push` with `host` and `job` labels.
- Reads journald logs from the host's `/run/systemd/journal/socket`.
- Reads container logs from `/var/lib/containers/storage/...` (podman).
- Forwards to Loki with `job` labels.
## Volumes
| Container path | Host path (convention) |
|----------------|------------------------|
| `/etc/alloy/config.alloy` | `shared/alloy/config.alloy` (bind mount) |
| Container path | Host path |
|----------------|-----------|
| `/etc/alloy` | `/opt/alloy/config` |
| `/run/systemd/journal/socket` | host journald socket (read-only) |
| `/var/log` | host logs (read-only) |
| `/var/lib/containers` | podman storage (read-only) |
## The UI port is published on purpose
Alloy's UI listens on the container's own `127.0.0.1:12345` by default, and Alloy 1.x refuses to configure the `ui` service from a config block ("service ui does not support being configured"). Under compose that meant **no reachable health endpoint at all**: the homepage status card for alloy read `down` continuously and a deploy had nothing to wait on. The unit therefore passes `--server.http.listen-addr=0.0.0.0:12345` and publishes `127.0.0.1:12345` — same posture as the other three: reachable from the host and from Caddy's network, and from nothing off-host. Alloy has no authentication, which is exactly why it is not published wider.
## Gotchas
- The journald socket mount is required for journal scraping.
- Container log paths differ between docker (`/var/lib/docker/containers`) and podman (`/var/lib/containers/storage/...`); point the config at the right one per host.
- One Alloy per host; do not run a single instance that tries to reach both hosts' logs.
- The unit `Wants=`/`After=` `loki.service`: without a reachable Loki the write client fails and the pipeline backs up.
## See Also
- [[Loki]] — the log backend
- [[Observability]] — the full stack
- [[Observability]] — the full stack, including what is still unconfigured