fix(monitoring): scrape node-dankclaw via host.containers.internal

node-exporter is net=host, so its 192.168.0.143:9100 bind is a host bind,
not a published port — the bridge-networked Prometheus container cannot
reach its own host's LAN IP there (netavark DNATs published ports only).
The node-dankclaw target has been red by construction since the
observability cutover while the container reported healthy. Point it at
host.containers.internal:9100 (netavark delivers that to the host LAN IP,
where node-exporter listens) — the same rule frank-large and frank-small
already use. node-dankserver is a different machine and keeps its LAN
address. Update the cross-file contract test and the Targets table.
This commit is contained in:
Frank committed 2026-09-21 11:09:03 -07:00
1 parent f998a2d9f2
commit 5c1006dda6
3 files changed
+31 -8

No files matched your search

+19 -4
View File
@@ -108,12 +108,27 @@ service_section() {
@test "prometheus scrapes node-exporter where node-exporter actually listens" {
# Cross-file contract: if either side's address changes alone, host metrics
# silently stop with no error anywhere.
#
# This Prometheus is a bridge container (Network=dankclaw_default), and on
# this netavark host a container can only reach the host's LAN IP at a
# PUBLISHED port. node-exporter is net=host, so its 192.168.0.143:9100 bind
# is a host bind, not a published port — unreachable at its own LAN address.
# The node-dankclaw job therefore scrapes host.containers.internal:9100,
# which netavark delivers to the host's LAN IP, where node-exporter listens.
# (node-dankserver is a different machine, so its LAN address is reached
# over the real network and is scraped directly.)
local unit="$UNIT_DIR/node-exporter.container"
local cfg="$REPO_ROOT/shared/prometheus/prometheus.yml"
local addr
addr=$(grep -oE '192\.168\.0\.[0-9]+:9100' "$unit" | head -1)
[ -n "$addr" ] || { echo "node-exporter has no LAN listen address"; return 1; }
grep -q "\"$addr\"" "$cfg" || { echo "prometheus does not scrape $addr"; return 1; }
local lan
lan=$(grep -oE '192\.168\.0\.[0-9]+:9100' "$unit" | head -1)
[ -n "$lan" ] || { echo "node-exporter has no LAN listen address"; return 1; }
# The exporter must still bind that LAN address — it is where the
# host.containers.internal delivery lands.
grep -q -- "--web.listen-address=$lan" "$unit" \
|| { echo "node-exporter no longer binds $lan"; return 1; }
# The local job scrapes via the gateway, not the (unreachable) LAN address.
grep -q '"host.containers.internal:9100"' "$cfg" \
|| { echo "node-dankclaw does not scrape host.containers.internal:9100"; return 1; }
}
@test "container names still match what Caddy and Prometheus address by name" {