feat: initialize serverAppConfigs with podman stacks for Dankserver and Dankclaw
- Dankserver stack: RomM, Copyparty, Calibre-Web, node-exporter - Dankclaw stack: Gluetun VPN (SearXNG tunnel), SearXNG+Valkey, ComfyUI (CPU backend), OpenWebUI, Hermes, Gitea+Postgres (migrated), Caddy reverse proxy, Grafana+Prometheus+Loki+Alloy observability - Caddyfiles with .local hostname convention and basicauth homepage - bind-mount storage policy, no named volumes - deploy.sh mirroring vllmScripts git workflow (push -> pull -> restart -> health) - docs/ Obsidian vault: per-service docs from frank-default-small research subagents (promtail EOL -> Alloy; gitea SECRET_KEY preservation; ComfyUI --cpu) - bats test suite at 6 seams — 22 tests green
This commit is contained in:
commit
e3eb5334cb
57 files changed
+3033
No files matched your search
@@ -0,0 +1,53 @@
|
||||
# Observability
|
||||
|
||||
The observability stack runs on Dankclaw: Grafana for dashboards, Prometheus for metrics, and log aggregation for container logs. This is a small homelab — no Kubernetes.
|
||||
|
||||
## Stack
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
S[Services] -->|/metrics| P[Prometheus]
|
||||
S -->|logs| T[Promtail]
|
||||
T --> L[Loki]
|
||||
P --> G[Grafana]
|
||||
L --> G
|
||||
```
|
||||
|
||||
| Component | Role | Image | Port |
|
||||
|-----------|------|-------|------|
|
||||
| [[Prometheus]] | metrics collection | `prom/prometheus` | 9090 |
|
||||
| [[Grafana]] | dashboards | `grafana/grafana` | 3002 |
|
||||
| Loki | log storage & query | `grafana/loki` | 3100 |
|
||||
| Alloy | log shipping (per host) | `grafana/alloy` | — |
|
||||
|
||||
## Log Aggregation Choice
|
||||
|
||||
Loki + Alloy is the lightweight Grafana-native option for a two-server homelab:
|
||||
|
||||
- Alloy reads container logs from the host (journald socket + container log files) and pushes them to Loki. It replaces Promtail, which reached end-of-life on 2026-03-02.
|
||||
- Loki stores compressed logs with labels; queries run in Grafana's Explore.
|
||||
- No Elasticsearch memory cost, no Kubernetes required.
|
||||
|
||||
Alternatives considered and rejected: ELK (too heavy), Promtail (EOL), VictoriaLogs (newer, less documentation), plain `journald` (no centralized query).
|
||||
|
||||
## Scrape Surface
|
||||
|
||||
| Target | Source |
|
||||
|--------|--------|
|
||||
| Prometheus self | `localhost:9090/metrics` |
|
||||
| ComfyUI | `/system_stats` (limited) or blackbox probe |
|
||||
| Gitea | `/metrics` (built-in exporter) |
|
||||
| Grafana | `/metrics` (built-in) |
|
||||
| Host metrics | node-exporter `:9100` |
|
||||
|
||||
For services without native exporters, use the Prometheus **blackbox exporter** in probe mode to assert up/down and HTTP latency. The homepage status also probes the same endpoints.
|
||||
|
||||
## Healthchecks
|
||||
|
||||
Every compose service gets a healthcheck so podman reports container health and Promtail/Grafana can alert on it. See [[Deployment Flow]] for the verify step.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[Grafana]], [[Prometheus]]
|
||||
- [[Dankclaw Stack]]
|
||||
- [[Homepage]] — the user-facing status surface
|
||||
Reference in new issue
Block a user