prometheus: drop the frank-primary scrape job (FreeToken exports no /metrics)

FreeToken has no /metrics endpoint — the job scraped 404s forever, a
structural never-green, not an outage (the Lucebox DS4.1 precedent). The
LlmEndpointDown alert regex self-adjusts (frank-.* matches existing series
only). Comments updated; seam now asserts frank-primary's absence. If
FreeToken grows /metrics, re-add the job.
This commit is contained in:
Frank
2026-10-05 19:38:14 -07:00
parent e2308dff71
commit 8618ec6dbe
3 changed files with 25 additions and 30 deletions
+9 -12
View File
@@ -315,31 +315,28 @@ PAIRS
@test "prometheus scrapes every LLM endpoint that exports metrics" {
# The LLM endpoints are the services whose behaviour this fleet argues about
# most (KV pressure, prefix-cache reuse, queue depth). All three export
# the llamacpp:* family since 2026-09-26, when frank-primary (Dankserver)
# went back to llama.cpp — the unsloth b11030-mix prebuilt serving
# Qwen3.8-Flash-Next Q8_0 — after two days of the Lucebox DS4.1 engine,
# which exported nothing (404) and had no job.
# most (KV pressure, prefix-cache reuse, queue depth). Both Dankclaw
# endpoints export the llamacpp:* family. frank-primary LEFT the set
# 2026-10-05: the FreeToken PR587 ROCm engine has NO /metrics endpoint —
# scraping it was a structural never-green (404), so the job was deleted
# rather than left vacuous (the Lucebox DS4.1 precedent, 2026-09-26).
# The two Dankclaw endpoints are HOST processes reached through
# host.containers.internal: this Prometheus is a bridge-networked
# container, and on this netavark host a container can only reach the
# host's LAN IP at a PUBLISHED port — a host-process bind like
# llama-server's 0.0.0.0:7770 is refused at 192.168.0.143:7770 while it
# answers at 169.254.1.2:7770 (why frank-small stayed red by construction
# for days until the 2026-09-20 /targets audit). frank-primary is a
# DIFFERENT machine, so that quirk does not apply: plain LAN routing
# reaches 192.168.0.235:7770, and the job must say so (a gateway target
# would point at Dankclaw's own :7770 — frank-large — and double-scrape
# it under the wrong label).
# for days until the 2026-09-20 /targets audit).
local cfg="$REPO_ROOT/shared/prometheus/prometheus.yml"
grep -q '"host.containers.internal:7771"' "$cfg" || { echo "frank-small not scraped"; return 1; }
grep -q '"host.containers.internal:7770"' "$cfg" || { echo "frank-large not scraped"; return 1; }
grep -q '"192.168.0.235:7770"' "$cfg" || { echo "frank-primary not scraped"; return 1; }
! grep -q '"192.168.0.235:7770"' "$cfg"
grep -q '"host.containers.internal:7770,192.168.0.235:7770"' "$cfg" \
&& { echo "frank-primary folded into a Dankclaw gateway target"; return 1; }
for job in frank-small frank-large frank-primary; do
for job in frank-small frank-large; do
grep -q "job_name: \"$job\"" "$cfg" || { echo "no scrape job for $job"; return 1; }
done
! grep -q 'job_name: "frank-primary"' "$cfg"
# None of them is reached through a guarded vhost: Prometheus cannot solve
# an Anubis challenge, so a franksdank.cloud target is red by construction.
! grep -qE '^\s*-\s*"?https?://[^"]*franksdank' "$cfg"