Files

T

Frank e3eb5334cb feat: initialize serverAppConfigs with podman stacks for Dankserver and Dankclaw

- Dankserver stack: RomM, Copyparty, Calibre-Web, node-exporter
- Dankclaw stack: Gluetun VPN (SearXNG tunnel), SearXNG+Valkey, ComfyUI
  (CPU backend), OpenWebUI, Hermes, Gitea+Postgres (migrated), Caddy
  reverse proxy, Grafana+Prometheus+Loki+Alloy observability
- Caddyfiles with .local hostname convention and basicauth homepage
- bind-mount storage policy, no named volumes
- deploy.sh mirroring vllmScripts git workflow (push -> pull -> restart -> health)
- docs/ Obsidian vault: per-service docs from frank-default-small research
  subagents (promtail EOL -> Alloy; gitea SECRET_KEY preservation; ComfyUI --cpu)
- bats test suite at 6 seams — 22 tests green

2026-08-13 20:19:41 -07:00

1.9 KiB

Raw Blame History

Observability

The observability stack runs on Dankclaw: Grafana for dashboards, Prometheus for metrics, and log aggregation for container logs. This is a small homelab — no Kubernetes.

Stack

graph LR
    S[Services] -->|/metrics| P[Prometheus]
    S -->|logs| T[Promtail]
    T --> L[Loki]
    P --> G[Grafana]
    L --> G

Component	Role	Image	Port
Prometheus	metrics collection	`prom/prometheus`	9090
Grafana	dashboards	`grafana/grafana`	3002
Loki	log storage & query	`grafana/loki`	3100
Alloy	log shipping (per host)	`grafana/alloy`	—

Log Aggregation Choice

Loki + Alloy is the lightweight Grafana-native option for a two-server homelab:

Alloy reads container logs from the host (journald socket + container log files) and pushes them to Loki. It replaces Promtail, which reached end-of-life on 2026-03-02.
Loki stores compressed logs with labels; queries run in Grafana's Explore.
No Elasticsearch memory cost, no Kubernetes required.

Alternatives considered and rejected: ELK (too heavy), Promtail (EOL), VictoriaLogs (newer, less documentation), plain journald (no centralized query).

Scrape Surface

Target	Source
Prometheus self	`localhost:9090/metrics`
ComfyUI	`/system_stats` (limited) or blackbox probe
Gitea	`/metrics` (built-in exporter)
Grafana	`/metrics` (built-in)
Host metrics	node-exporter `:9100`

For services without native exporters, use the Prometheus blackbox exporter in probe mode to assert up/down and HTTP latency. The homepage status also probes the same endpoints.

Healthchecks

Every compose service gets a healthcheck so podman reports container health and Promtail/Grafana can alert on it. See Deployment Flow for the verify step.

1.9 KiB Raw Blame History