feat(monitoring,harness): follow frank-medium onto vllm:* metrics and fix red-by-construction targets
The radiance TP=2 cutover changed the metric namespace frank-medium exports (llamacpp:* -> vllm:*) and left two scrape targets addressing host processes by an address a container cannot reach. - shared/prometheus/rules/llm-fleet.yml: two namespaces side by side. Added LlmRequestsWaiting, LlmKvCacheSaturated (vllm-only gauges llama.cpp never had), the job:llm_vllm_prefix_cache_reuse:rate5m and job:llm_spec_accept_length:rate5m recording rules, and re-cut LlmPrefixCacheCollapse / LlmSpecDecodeOff to OR both families with per-family traffic floors so an idle endpoint cannot fire them. Every expression validated against live Prometheus. - shared/prometheus/prometheus.yml: frank-small -> host.containers.internal:7771 and frank-large scraped at host.containers.internal:7770. Both are HOST processes on Dankclaw and netavark DNATs published container ports only, so the LAN-IP targets were red by construction; frank-large's CIRU llama.cpp build does export the llamacpp:* family, contra the old comment. - tests/observability_quadlet.bats: the two seams now encode the new contract (per-endpoint reachability by process type; every named series must be on the measured list for its engine; engine-independent rules must name both). - homepage docs.html/services.json: frank-medium sees pictures, frank-large is CIRU Orca with a single seat (was halogen w4b, two seats, 200k). - docs: radiance runbook is ACTIVE (deployed flags, three-arm battery, verified capabilities, rollback, replica-era sections labelled); llama.cpp runbook carries the SUPERSEDED banner; Dual-R9700-Cooperation states the measured verdict; Observability records the namespace split and the DNAT lesson; AGENTS.md topology/frank-medium/Model Config/Benchmarking Policy/runbook index flipped; both indexes updated; work-log entry added. Also repaired agent-pass damage: prose prepended above frontmatter openers, duplicated runbook bodies, a literal tool-call token pasted into a runbook, and an index destroyed by diff artifacts (restored from HEAD, flips re-applied). Vault link check: 128 pages, 0 unresolved. ./tests/run.sh: 325 pass.
This commit is contained in:
1 parent
c2116658f8
commit
08f1c7ad09
13 files changed
+687
-286
No files matched your search
@@ -47,10 +47,10 @@
|
||||
|
||||
<h2>frank-large — the big brain</h2>
|
||||
<ul>
|
||||
<li><strong>Model:</strong> Qwen3.8-Flash-Next, w4b quantization, served by halogen-flash-server.</li>
|
||||
<li><strong>Model:</strong> Qwen3.8-Flash CIRU-STRIX-Orca, served by CIRU v4.4.1 llama.cpp with the model's own MTP draft head.</li>
|
||||
<li><strong>Hardware:</strong> Dankclaw's Strix Halo iGPU (gfx1151), shared with the whole host.</li>
|
||||
<li><strong>Best for:</strong> long, thoughtful text conversations. 200k-token context per conversation.</li>
|
||||
<li><strong>Capacity:</strong> exactly two conversations resident at once. A third waits — politely, but it waits.</li>
|
||||
<li><strong>Best for:</strong> long, thoughtful text conversations at the model's full trained window.</li>
|
||||
<li><strong>Capacity:</strong> exactly one conversation resident at a time. A second waits — politely, but it waits.</li>
|
||||
<li><strong>Not for:</strong> images. It cannot see them.</li>
|
||||
</ul>
|
||||
|
||||
@@ -67,11 +67,11 @@
|
||||
<li><strong>Model:</strong> Qwen3.8-27B-Uncensored at MXFP4, on vllm-radiance with a DFlash2 draft head.</li>
|
||||
<li><strong>Hardware:</strong> Dankserver's two Radeon R9700s as ONE tensor-parallel engine (TP=2).</li>
|
||||
<li><strong>Best for:</strong> general chat and reasoning when the big brain is busy; the biggest pure-language model we run, and the fastest we have measured.</li>
|
||||
<li><strong>Capacity:</strong> 262k-token context, shared by up to eight conversations at once. Text only — pictures go to <code>frank-small</code>.</li>
|
||||
<li><strong>Capacity:</strong> 262k-token context shared by up to eight conversations at once. It can see pictures too; <code>frank-small</code> is the faster eye and holds four conversations.</li>
|
||||
</ul>
|
||||
|
||||
<p class="foot-note">Which one to pick? Screenshots and photos: <code>frank-small</code>.
|
||||
Deep long-text work: <code>frank-large</code> (mind the two-seat limit).
|
||||
Deep long-text work: <code>frank-large</code> (mind the single seat).
|
||||
When in doubt: <code>frank-medium</code>.</p>
|
||||
</section>
|
||||
|
||||
|
||||
@@ -233,7 +233,7 @@
|
||||
{
|
||||
"name": "frank-large",
|
||||
"category": "user",
|
||||
"description": "The big brain. Qwen3.8-Flash-Next w4b on halogen-flash-server, running on Dankclaw's Strix Halo iGPU. Text-only; holds two 200k-token conversations resident.",
|
||||
"description": "The big brain. Qwen3.8-Flash CIRU-STRIX-Orca on CIRU v4.4.1 llama.cpp, running on Dankclaw's Strix Halo iGPU. Text-only; one conversation resident at a time.",
|
||||
"url": "",
|
||||
"docs": "/docs#models",
|
||||
"probe": "127.0.0.1:7770",
|
||||
|
||||
Reference in new issue
Block a user