Frank
8500600d09
The 2026-09-26 same-day cutover that ended the Lucebox DS4.1 detour:
Dankserver's :7770 endpoint is now unsloth llama.cpp (the b11030-mix
prebuilt, ROCm gfx120X, bundled runtime) serving
unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 — 175.30 GiB across six sha256-
verified split parts — on both Radeon AI PRO R9700s, as a ROOT unit (the
frank-large shape: no container, no generated unit).
The deployed contract:
- Placement law: layer split -ts 1,1 with -ngl 999; the routed experts
and the 51B-param n-gram (PLE) table are COMPUTED in host RAM via
-ot exps=CPU,ngram_embeddings=CPU (the x4-pair measurement: streaming
over PCIe loses at 3.7 t/s, computing in RAM does not care).
per_layer_token_embd.weight=CPU is required by the shared MTP head's
borrow path (a GPU-resident borrow crashes the draft load).
mmap stays on; --cache-ram 16384.
- Spec chain draft-mtp,ngram-mod: the shared Q8_0 MTP head
(mtp-...-shared-Q8_0.gguf, 2,786,568,256 B) at depth 2, with the
draftless ngram-mod matcher chained behind it — the only legal way to
layer two spec implementations in one process.
- Vision explicit: --mmproj names the BF16 projector (this build never
autoloads; a server without it comes up healthy and text-only).
- Reasoning: --reasoning-format deepseek → reasoning_content; the
Qwen3.8 template's {xhigh, medium, low} reasoning_effort gate folded
by reasoningEffortMap in models.yml.
- One stream, the model's full 262,144 context; greedy server default
with Qwen-recommended sampling riding compat.extraBody.
The deployer's deploy_llm_primary is rewritten for the root unit: it
gates every artifact at its declared byte count (stat -c %s, never wc -c
— which read the whole 377 GiB lucebox file for minutes on 2026-09-26),
retires the lucebox user quadlet to dated home trash, waits out port
7770, relinks/enables/restarts the root unit and waits up to 1800 s for
/health. The quadlet-era tree under dankserver/frank-primary/ is
archived (dankserver/.AGENTTRASH/20260926-llamacpp-cutover/ on this
box; git history everywhere else) and its planner mapping replaced by
the scripts/ trio.
Observability flips back to the llamacpp namespace: a frank-primary
Prometheus job at 192.168.0.235:7770 (cross-host LAN — the netavark
published-port quirk that forces host.containers.internal for the two
Dankclaw endpoints is same-host only), the LlmEndpointDown runbook
text, and the dashboard description for three llama.cpp endpoints.
Docs: the new runbook (frank-primary-llamacpp-qwen38-flash-next-q8.md),
RETIRED banners on the lucebox and radiance runbooks and their index
entries, the AGENTS.md fleet section rewritten for the root-unit shape,
and the homepage frank-medium card renamed to frank-primary with the
Q8_0 Flash-Next spec (docs.html #models and the modality note updated:
images on frank-primary and frank-small, audio on frank-small only).
models.yml: the dankserver provider is "llamacpp (dankserver)" serving
frank-primary, contextWindow 262144, maxTokens 65536 (compaction-gap
law: 183,501 + 65,536 <= 262,144), input [text, image].
Suite: 346 green (frank_primary_unit.bats rewritten for the root unit —
16 tests pinning the placement law, the shared-head requirement, the
explicit projector, HIP env hygiene (BIND_HOST, never bare HOST), and
the deployer's byte-count gates; the quadlet-era planner and
changed-tests mappings rewritten for the trio; the observability scrape
test asserts three llamacpp jobs again).
Rollback: the lucebox image (localhost/lucebox-rocm:gfx1201) and the
378 GiB payload stay on the host; the Qwen payload stays fetched, so a
re-swap in either direction is a redeploy, not a refetch.
125 lines
4.6 KiB
YAML
125 lines
4.6 KiB
YAML
# Prometheus configuration for the homelab.
|
|
# Deployed to /opt/prometheus/config/prometheus.yml on Dankclaw.
|
|
|
|
global:
|
|
scrape_interval: 15s
|
|
scrape_timeout: 10s
|
|
evaluation_interval: 15s
|
|
|
|
# Alerting and recording rules for the LLM fleet. They evaluate, expose
|
|
# state through /alerts and the ALERTS metric, and — since 2026-09-17 —
|
|
# deliver to Signal through Alertmanager (see the alerting section below).
|
|
rule_files:
|
|
- /etc/prometheus/rules/*.yml
|
|
|
|
# The delivery half. Alertmanager runs on the same dankclaw_default network,
|
|
# so it is addressed by container name, like grafana and loki below. Its
|
|
# single receiver webhooks the host-side signal-relay (192.168.0.143:18093,
|
|
# reached from the container as host.containers.internal).
|
|
alerting:
|
|
alertmanagers:
|
|
- static_configs:
|
|
- targets: ["alertmanager:9093"]
|
|
|
|
scrape_configs:
|
|
# Prometheus itself
|
|
- job_name: "prometheus"
|
|
static_configs:
|
|
- targets: ["localhost:9090"]
|
|
|
|
# Host metrics — both hosts. node-dankclaw is scraped over
|
|
# host.containers.internal, not its own LAN address: this Prometheus is a
|
|
# bridge container, and on this netavark host a container can only reach the
|
|
# host's LAN IP at a PUBLISHED port — node-exporter's net=host bind on
|
|
# 192.168.0.143:9100 is refused by construction (the same rule that sends
|
|
# frank-large and frank-small to the gateway below). node-dankserver is a
|
|
# different machine, so its LAN address works.
|
|
- job_name: "node-dankclaw"
|
|
static_configs:
|
|
- targets: ["host.containers.internal:9100"]
|
|
|
|
- job_name: "node-dankserver"
|
|
static_configs:
|
|
- targets: ["192.168.0.235:9100"]
|
|
|
|
# Gitea metrics (quadlet on an isolated network; use the published LAN port)
|
|
- job_name: "gitea"
|
|
static_configs:
|
|
- targets: ["192.168.0.143:3000"]
|
|
metrics_path: "/metrics"
|
|
|
|
# Grafana metrics
|
|
- job_name: "grafana"
|
|
static_configs:
|
|
- targets: ["grafana:3000"]
|
|
metrics_path: "/metrics"
|
|
|
|
# Loki metrics
|
|
- job_name: "loki"
|
|
static_configs:
|
|
- targets: ["loki:3100"]
|
|
metrics_path: "/metrics"
|
|
|
|
# ComfyUI (no native /metrics; probe the UI)
|
|
- job_name: "comfyui"
|
|
metrics_path: "/"
|
|
static_configs:
|
|
- targets: ["comfyui:8188"]
|
|
|
|
# Blackbox-style probes for service up/down (published LAN ports;
|
|
# gitea and comfyui are covered by their own jobs above)
|
|
- job_name: "services"
|
|
metrics_path: "/"
|
|
static_configs:
|
|
- targets:
|
|
- "192.168.0.143:8081" # SearXNG (via the Gluetun tunnel)
|
|
- "192.168.0.143:3005" # OpenWebUI
|
|
|
|
# LLM fleet. Scraped on the LAN addresses where the endpoint is reachable from
|
|
# this container, not the guarded vhosts: these are machine clients and
|
|
# Prometheus cannot solve an Anubis challenge.
|
|
#
|
|
# One metric namespace: llamacpp:* — frank-large, frank-small and
|
|
# frank-primary, all llama.cpp with /metrics (the family verified from
|
|
# /metrics on 2026-09-16 and 2026-09-20) and all exporting
|
|
# requests_deferred (the queue depth, the only form llama.cpp shows the
|
|
# capacity wall in), spec_decode_num_drafts_total, tokens_predicted_total,
|
|
# and the prompt_tokens_cached_total / prompt_tokens_total reuse pair.
|
|
#
|
|
# frank-large and frank-small are scraped over host.containers.internal,
|
|
# not their own LAN addresses: this Prometheus is a bridge-networked
|
|
# container (Network=dankclaw_default), and on this netavark host a
|
|
# container can only reach the host's LAN IP at a PUBLISHED port — a
|
|
# host-process bind like llama-server's 0.0.0.0:7770 is refused at
|
|
# 192.168.0.143:7770 while it answers at 169.254.1.2:7770. That is
|
|
# why frank-small was red here for days (up{job="frank-small"} == 0,
|
|
# "dial tcp 192.168.0.143:7771") even though the endpoint was healthy and
|
|
# exporting: the target address was unreachable by construction, not the
|
|
# service down. Same host, same rule, so both local endpoints use the
|
|
# gateway.
|
|
#
|
|
# (2026-09-26) frank-primary is llama.cpp again (the unsloth b11030-mix
|
|
# prebuilt serving Qwen3.8-Flash-Next Q8_0) on Dankserver, a DIFFERENT
|
|
# machine: the netavark published-port quirk above is same-host only, so
|
|
# plain LAN routing reaches 192.168.0.235:7770 and the job returns.
|
|
- job_name: "frank-small"
|
|
metrics_path: "/metrics"
|
|
static_configs:
|
|
- targets: ["host.containers.internal:7771"]
|
|
labels:
|
|
service: frank-small
|
|
|
|
- job_name: "frank-large"
|
|
metrics_path: "/metrics"
|
|
static_configs:
|
|
- targets: ["host.containers.internal:7770"]
|
|
labels:
|
|
service: frank-large
|
|
|
|
- job_name: "frank-primary"
|
|
metrics_path: "/metrics"
|
|
static_configs:
|
|
- targets: ["192.168.0.235:7770"]
|
|
labels:
|
|
service: frank-primary
|