From 7a8cc83385b4d6b9067f4aa42c94c1f3829fcf7d Mon Sep 17 00:00:00 2001 From: Frank Date: Wed, 21 Oct 2026 16:32:36 -0700 Subject: [PATCH] fix(quadlet): drop HostAddr from prometheus.unit - podman 5.7.0 rejects the key and the quadlet generator then aborts the ENTIRE user batch, silently orphaning every other unit (prometheus.service vanished while the container ran; deploy died 'Unit prometheus.service not found', live 2026-10-21). host.containers.internal resolves natively; the key was never needed. --- .../observability/quadlet/prometheus.container | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/dankclaw/observability/quadlet/prometheus.container b/dankclaw/observability/quadlet/prometheus.container index bbbb125..7bebfc4 100644 --- a/dankclaw/observability/quadlet/prometheus.container +++ b/dankclaw/observability/quadlet/prometheus.container @@ -43,14 +43,16 @@ Wants=network-online.target [Container] ContainerName=prometheus -Image=docker.io/prom/prometheus:latest +Image=docker.io/prom/prometheus:latest@sha256:fa3302279710be2f21a8301ec17a8b4f2301fd829bd113839b3b6b02b6f191a2 Network=dankclaw_default -# Name the podman gateway so scrape jobs can reach same-host targets (the -# netavark hairpin gap bars this container from dialing its own host's LAN -# IP; the cadvisor/gpu jobs and the blackbox scrape all ride this name). -# Same mechanism the hermes Caddy upstream learned the hard way (Container -# Host Reachability in the vault). -HostAddr=host.containers.internal=169.254.1.2 +# Same-host scrape targets are dialed at host.containers.internal (the podman +# gateway): a bridge container cannot dial its own host's LAN IP (netavark +# hairpin gap, measured 2026-10-21), and traffic to the gateway lands on host +# loopback / published ports (cadvisor/gpu jobs and the blackbox scrape all +# ride it). The name resolves natively — NO HostAddr key: quadlet 5.7.0 calls +# it unsupported and the WHOLE user quadlet batch then fails to generate, +# silently orphaning every other unit (live 2026-10-21: prometheus.service +# vanished while its container kept running as an orphan). User=0:0 Exec=--config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus --storage.tsdb.retention.time=30d --web.enable-lifecycle PublishPort=127.0.0.1:9090:9090