Frank
701ab14f60
Rewrites the [[Observability]] umbrella page and the five component pages for what is actually deployed: rootless Quadlet units instead of compose services, the loopback-only publish posture (node-exporter's LAN binding explained), the loki healthcheck living in its image because the image has no shell, and the Home=/root trap that killed the first grafana start. Adds the note that matters most to a reader: the stack is installed, healthy and UNCONFIGURED. Grafana reports zero datasources and zero dashboards with an empty provisioning directory; Prometheus has no rule files and no Alertmanager; Loki is configured but has no reader; node-exporter runs its default collectors. Alloy is the only configured piece. Metrics and logs are collected and retained, and nothing is visualized or alerted on. AGENTS.md's boot-persistence gap note is replaced by the resolution.
55 lines
2.6 KiB
Markdown
55 lines
2.6 KiB
Markdown
# Loki
|
|
|
|
Loki is the log aggregation backend of the observability stack. It stores and queries container and system logs. It replaces the need for a full ELK stack.
|
|
|
|
**Host:** Dankclaw — **Port:** 3100 (loopback) — **Docs:** https://grafana.com/docs/loki/latest
|
|
|
|
## Facts
|
|
|
|
| Item | Value |
|
|
|------|-------|
|
|
| Image | `franksdank/loki:latest` (local build: upstream + a static busybox for the healthcheck) |
|
|
| Published | `127.0.0.1:3100` → 3100, loopback only |
|
|
| Unit | `dankclaw/observability/quadlet/loki.container` → `loki.service` |
|
|
| Mode | single-binary (all components in one process) |
|
|
| Storage | filesystem, `/opt/loki/data` |
|
|
| Auth | `auth_enabled: false` (single-tenant homelab) |
|
|
| Schema | v13 / TSDB |
|
|
| Retention | 30 days, enforced by the compactor |
|
|
|
|
## Configuration
|
|
|
|
`shared/loki/loki-config.yaml` is the source of truth, installed to `/opt/loki/config/loki-config.yaml` on every deploy. Key settings:
|
|
|
|
- `schema_config` v13 with `store: tsdb` and a filesystem object store.
|
|
- `compactor` with `retention_enabled: true` so logs do not grow without bound.
|
|
- `replication_factor: 1` for a single instance.
|
|
|
|
## Volumes
|
|
|
|
| Container path | Host path |
|
|
|----------------|-----------|
|
|
| `/loki` | `/opt/loki/data` |
|
|
| `/etc/loki` | `/opt/loki/config` |
|
|
|
|
## Healthcheck
|
|
|
|
Loki responds on `/ready` with 200. The health command is **not** in the unit: a unit `HealthCmd` reaches podman as `CMD-SHELL`, i.e. `/bin/sh -c`, and this image is distroless with no shell at all. It ships in `dankclaw/loki/Containerfile` as an exec-form `HEALTHCHECK` instead (which is why the image carries busybox, and why the build must pass `--format docker` — the OCI image format has no `HEALTHCHECK` field and silently drops it).
|
|
|
|
## Log Shipping
|
|
|
|
The [[Alloy]] agent on Dankclaw ships container and journal logs to `http://loki:3100/loki/api/v1/push` — by container name on the `dankclaw_default` network, never through the vhost (a non-browser client cannot solve the Anubis challenge).
|
|
|
|
## Gotchas
|
|
|
|
- Filesystem storage grows; keep compactor retention on.
|
|
- Loki and Alloy must agree on the push URL: `http://loki:3100/loki/api/v1/push`.
|
|
- The listener is loopback-only now. Shipping logs from a second host would need the unit's publish changed (or a shared network), not just an Alloy install — Loki has no authentication of its own, so that decision is a security one, not a config one.
|
|
- Nothing queries Loki yet: Grafana has no datasource configured. See the state note in [[Observability]].
|
|
|
|
## See Also
|
|
|
|
- [[Alloy]] — the log shipper
|
|
- [[Grafana]] — the query frontend
|
|
- [[Observability]] — the whole stack
|