docs: rename docs/ -> serverAppConfigs-docs/ per system-wide PROJECTNAME-docs convention
All in-repo path references (AGENTS.md, README, old scripts, report artifacts,
test code paths) rewritten in the same commit; focused-test map extended for
.gitignore, old/scripts/*, dankclaw/comfyui/*, and report/* per this repo's
own convention ('add one before committing'). Obsidian wikilinks are
title-based and unaffected.
This commit is contained in:
1 parent
6d4a96a423
commit
5829a0bfe8
140 files changed
+1820
-86
No files matched your search
@@ -0,0 +1,53 @@
|
||||
# Observability
|
||||
|
||||
The observability stack runs on Dankclaw: Grafana for dashboards, Prometheus for metrics, and log aggregation for container logs. This is a small homelab — no Kubernetes.
|
||||
|
||||
## Stack
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
S[Services] -->|/metrics| P[Prometheus]
|
||||
S -->|logs| T[Alloy]
|
||||
T --> L[Loki]
|
||||
P --> G[Grafana]
|
||||
L --> G
|
||||
```
|
||||
|
||||
| Component | Role | Image | Port |
|
||||
|-----------|------|-------|------|
|
||||
| [[Prometheus]] | metrics collection | `prom/prometheus` | 9090 |
|
||||
| [[Grafana]] | dashboards | `grafana/grafana` | 3002 |
|
||||
| Loki | log storage & query | `grafana/loki` | 3100 |
|
||||
| Alloy | log shipping (per host) | `grafana/alloy` | — |
|
||||
|
||||
## Log Aggregation Choice
|
||||
|
||||
Loki + Alloy is the lightweight Grafana-native option for a two-server homelab:
|
||||
|
||||
- Alloy reads container logs from the host (journald socket + container log files) and pushes them to Loki. It replaces Promtail, which reached end-of-life on 2026-03-02.
|
||||
- Loki stores compressed logs with labels; queries run in Grafana's Explore.
|
||||
- No Elasticsearch memory cost, no Kubernetes required.
|
||||
|
||||
Alternatives considered and rejected: ELK (too heavy), Promtail (EOL), VictoriaLogs (newer, less documentation), plain `journald` (no centralized query).
|
||||
|
||||
## Scrape Surface
|
||||
|
||||
| Target | Source |
|
||||
|--------|--------|
|
||||
| Prometheus self | `localhost:9090/metrics` |
|
||||
| ComfyUI | `/system_stats` (limited) or blackbox probe |
|
||||
| Gitea | `/metrics` (built-in exporter) |
|
||||
| Grafana | `/metrics` (built-in) |
|
||||
| Host metrics | node-exporter `:9100` |
|
||||
|
||||
For services without native exporters, use the Prometheus **blackbox exporter** in probe mode to assert up/down and HTTP latency. The homepage status also probes the same endpoints.
|
||||
|
||||
## Healthchecks
|
||||
|
||||
Every compose service gets a healthcheck so podman reports container health and Promtail/Grafana can alert on it. See [[Deployment Flow]] for the verify step.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[Grafana]], [[Prometheus]]
|
||||
- [[Dankclaw Stack]]
|
||||
- [[Homepage]] — the user-facing status surface
|
||||
Reference in new issue
Block a user