docs: work-log entry for the node-dankclaw scrape fix
This commit is contained in:
@@ -11,6 +11,29 @@ timestamp: 2026-09-14
|
||||
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-09-21 — node-dankclaw Prometheus target fixed (netavark host-bind quirk)
|
||||
|
||||
- **The gap.** A deep audit found the `node-dankclaw` scrape target red while
|
||||
its container reported healthy. node-exporter runs `Network=host`, so its
|
||||
`192.168.0.143:9100` bind is a host bind, not a published port — and this
|
||||
Prometheus is a bridge container (`Network=dankclaw_default`) that, on this
|
||||
netavark host, can only reach the host's LAN IP at a PUBLISHED port. The
|
||||
target was unreachable by construction, not the exporter down. This was a
|
||||
fourth down job the 2026-09-16 "three jobs read down" audit did not list.
|
||||
- **The fix.** Scrape `host.containers.internal:9100` instead — netavark
|
||||
delivers that address to the host's LAN IP, where node-exporter listens.
|
||||
The same rule `frank-large` and `frank-small` already use (documented in the
|
||||
LLM comment block of `shared/prometheus/prometheus.yml`). `node-dankserver`
|
||||
is a different machine and keeps its LAN address. Verified live: the
|
||||
workaround returns real metrics from inside the container, and after deploy
|
||||
`up{job="node-dankclaw"}` is 1 — the down set is back to exactly the three
|
||||
structural jobs (comfyui, services, gitea).
|
||||
- **Files.** `shared/prometheus/prometheus.yml` (target + comment),
|
||||
`serverAppConfigs-docs/Infrastructure/Prometheus.md` (Targets table), and the
|
||||
cross-file contract test `tests/observability_quadlet.bats` (now asserts the
|
||||
exporter binds the LAN address AND the local job scrapes via the gateway,
|
||||
instead of asserting scrape-target == LAN address, which the quirk breaks).
|
||||
|
||||
## 2026-09-20 — DFlash2 vs MTP A/B on the FP8 checkpoint: DFlash2 wins on decode
|
||||
|
||||
- **The test.** The FP8 checkpoint carries its own single-layer MTP head, so
|
||||
|
||||
Reference in New Issue
Block a user