diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index 1cb2d90..2bfa9c5 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -11,6 +11,29 @@ timestamp: 2026-09-14 # Bundle Change Log +## 2026-09-21 — node-dankclaw Prometheus target fixed (netavark host-bind quirk) + +- **The gap.** A deep audit found the `node-dankclaw` scrape target red while + its container reported healthy. node-exporter runs `Network=host`, so its + `192.168.0.143:9100` bind is a host bind, not a published port — and this + Prometheus is a bridge container (`Network=dankclaw_default`) that, on this + netavark host, can only reach the host's LAN IP at a PUBLISHED port. The + target was unreachable by construction, not the exporter down. This was a + fourth down job the 2026-09-16 "three jobs read down" audit did not list. +- **The fix.** Scrape `host.containers.internal:9100` instead — netavark + delivers that address to the host's LAN IP, where node-exporter listens. + The same rule `frank-large` and `frank-small` already use (documented in the + LLM comment block of `shared/prometheus/prometheus.yml`). `node-dankserver` + is a different machine and keeps its LAN address. Verified live: the + workaround returns real metrics from inside the container, and after deploy + `up{job="node-dankclaw"}` is 1 — the down set is back to exactly the three + structural jobs (comfyui, services, gitea). +- **Files.** `shared/prometheus/prometheus.yml` (target + comment), + `serverAppConfigs-docs/Infrastructure/Prometheus.md` (Targets table), and the + cross-file contract test `tests/observability_quadlet.bats` (now asserts the + exporter binds the LAN address AND the local job scrapes via the gateway, + instead of asserting scrape-target == LAN address, which the quirk breaks). + ## 2026-09-20 — DFlash2 vs MTP A/B on the FP8 checkpoint: DFlash2 wins on decode - **The test.** The FP8 checkpoint carries its own single-layer MTP head, so