feat(monitoring,harness): follow frank-medium onto vllm:* metrics and fix red-by-construction targets

The radiance TP=2 cutover changed the metric namespace frank-medium exports
(llamacpp:* -> vllm:*) and left two scrape targets addressing host processes
by an address a container cannot reach.

- shared/prometheus/rules/llm-fleet.yml: two namespaces side by side. Added
  LlmRequestsWaiting, LlmKvCacheSaturated (vllm-only gauges llama.cpp never
  had), the job:llm_vllm_prefix_cache_reuse:rate5m and
  job:llm_spec_accept_length:rate5m recording rules, and re-cut
  LlmPrefixCacheCollapse / LlmSpecDecodeOff to OR both families with
  per-family traffic floors so an idle endpoint cannot fire them. Every
  expression validated against live Prometheus.
- shared/prometheus/prometheus.yml: frank-small -> host.containers.internal:7771
  and frank-large scraped at host.containers.internal:7770. Both are HOST
  processes on Dankclaw and netavark DNATs published container ports only, so
  the LAN-IP targets were red by construction; frank-large's CIRU llama.cpp
  build does export the llamacpp:* family, contra the old comment.
- tests/observability_quadlet.bats: the two seams now encode the new contract
  (per-endpoint reachability by process type; every named series must be on the
  measured list for its engine; engine-independent rules must name both).
- homepage docs.html/services.json: frank-medium sees pictures, frank-large is
  CIRU Orca with a single seat (was halogen w4b, two seats, 200k).
- docs: radiance runbook is ACTIVE (deployed flags, three-arm battery, verified
  capabilities, rollback, replica-era sections labelled); llama.cpp runbook
  carries the SUPERSEDED banner; Dual-R9700-Cooperation states the measured
  verdict; Observability records the namespace split and the DNAT lesson;
  AGENTS.md topology/frank-medium/Model Config/Benchmarking Policy/runbook
  index flipped; both indexes updated; work-log entry added.

Also repaired agent-pass damage: prose prepended above frontmatter openers,
duplicated runbook bodies, a literal tool-call token pasted into a runbook, and
an index destroyed by diff artifacts (restored from HEAD, flips re-applied).
Vault link check: 128 pages, 0 unresolved. ./tests/run.sh: 325 pass.
This commit is contained in:
Frank committed 2026-09-20 19:35:56 -07:00
1 parent c2116658f8
commit 08f1c7ad09
13 files changed
+687 -286

No files matched your search

+5 -5
View File
@@ -47,10 +47,10 @@
<h2>frank-large — the big brain</h2>
<ul>
<li><strong>Model:</strong> Qwen3.8-Flash-Next, w4b quantization, served by halogen-flash-server.</li>
<li><strong>Model:</strong> Qwen3.8-Flash CIRU-STRIX-Orca, served by CIRU v4.4.1 llama.cpp with the model's own MTP draft head.</li>
<li><strong>Hardware:</strong> Dankclaw's Strix Halo iGPU (gfx1151), shared with the whole host.</li>
<li><strong>Best for:</strong> long, thoughtful text conversations. 200k-token context per conversation.</li>
<li><strong>Capacity:</strong> exactly two conversations resident at once. A third waits — politely, but it waits.</li>
<li><strong>Best for:</strong> long, thoughtful text conversations at the model's full trained window.</li>
<li><strong>Capacity:</strong> exactly one conversation resident at a time. A second waits — politely, but it waits.</li>
<li><strong>Not for:</strong> images. It cannot see them.</li>
</ul>
@@ -67,11 +67,11 @@
<li><strong>Model:</strong> Qwen3.8-27B-Uncensored at MXFP4, on vllm-radiance with a DFlash2 draft head.</li>
<li><strong>Hardware:</strong> Dankserver's two Radeon R9700s as ONE tensor-parallel engine (TP=2).</li>
<li><strong>Best for:</strong> general chat and reasoning when the big brain is busy; the biggest pure-language model we run, and the fastest we have measured.</li>
<li><strong>Capacity:</strong> 262k-token context, shared by up to eight conversations at once. Text only — pictures go to <code>frank-small</code>.</li>
<li><strong>Capacity:</strong> 262k-token context shared by up to eight conversations at once. It can see pictures too; <code>frank-small</code> is the faster eye and holds four conversations.</li>
</ul>
<p class="foot-note">Which one to pick? Screenshots and photos: <code>frank-small</code>.
Deep long-text work: <code>frank-large</code> (mind the two-seat limit).
Deep long-text work: <code>frank-large</code> (mind the single seat).
When in doubt: <code>frank-medium</code>.</p>
</section>
+1 -1
View File
@@ -233,7 +233,7 @@
{
"name": "frank-large",
"category": "user",
"description": "The big brain. Qwen3.8-Flash-Next w4b on halogen-flash-server, running on Dankclaw's Strix Halo iGPU. Text-only; holds two 200k-token conversations resident.",
"description": "The big brain. Qwen3.8-Flash CIRU-STRIX-Orca on CIRU v4.4.1 llama.cpp, running on Dankclaw's Strix Halo iGPU. Text-only; one conversation resident at a time.",
"url": "",
"docs": "/docs#models",
"probe": "127.0.0.1:7770",