feat(monitoring,harness): follow frank-medium onto vllm:* metrics and fix red-by-construction targets

The radiance TP=2 cutover changed the metric namespace frank-medium exports
(llamacpp:* -> vllm:*) and left two scrape targets addressing host processes
by an address a container cannot reach.

- shared/prometheus/rules/llm-fleet.yml: two namespaces side by side. Added
  LlmRequestsWaiting, LlmKvCacheSaturated (vllm-only gauges llama.cpp never
  had), the job:llm_vllm_prefix_cache_reuse:rate5m and
  job:llm_spec_accept_length:rate5m recording rules, and re-cut
  LlmPrefixCacheCollapse / LlmSpecDecodeOff to OR both families with
  per-family traffic floors so an idle endpoint cannot fire them. Every
  expression validated against live Prometheus.
- shared/prometheus/prometheus.yml: frank-small -> host.containers.internal:7771
  and frank-large scraped at host.containers.internal:7770. Both are HOST
  processes on Dankclaw and netavark DNATs published container ports only, so
  the LAN-IP targets were red by construction; frank-large's CIRU llama.cpp
  build does export the llamacpp:* family, contra the old comment.
- tests/observability_quadlet.bats: the two seams now encode the new contract
  (per-endpoint reachability by process type; every named series must be on the
  measured list for its engine; engine-independent rules must name both).
- homepage docs.html/services.json: frank-medium sees pictures, frank-large is
  CIRU Orca with a single seat (was halogen w4b, two seats, 200k).
- docs: radiance runbook is ACTIVE (deployed flags, three-arm battery, verified
  capabilities, rollback, replica-era sections labelled); llama.cpp runbook
  carries the SUPERSEDED banner; Dual-R9700-Cooperation states the measured
  verdict; Observability records the namespace split and the DNAT lesson;
  AGENTS.md topology/frank-medium/Model Config/Benchmarking Policy/runbook
  index flipped; both indexes updated; work-log entry added.

Also repaired agent-pass damage: prose prepended above frontmatter openers,
duplicated runbook bodies, a literal tool-call token pasted into a runbook, and
an index destroyed by diff artifacts (restored from HEAD, flips re-applied).
Vault link check: 128 pages, 0 unresolved. ./tests/run.sh: 325 pass.
This commit is contained in:
Frank committed 2026-09-20 19:35:56 -07:00
1 parent c2116658f8
commit 08f1c7ad09
13 files changed
+687 -286

No files matched your search

@@ -1,7 +1,7 @@
---
type: configuration
title: frank-medium llama.cpp Runbook (DFlash2 drafter)
description: The ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
description: The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
tags:
- frank-medium
- dankserver
@@ -12,12 +12,17 @@ tags:
- gfx1201
- r9700
- layer-split
timestamp: 2026-09-19
timestamp: 2026-09-20
---
> **SUPERSEDED 2026-09-20.** Replaced by [[frank-medium-radiance-mxfp4]]
> (radiance TP=2, measured faster on every axis). The layer-split shape below is
> the rollback target, and the llama.cpp measurements in this page stay valid for
> it — but nothing here is the deployed shape any more.
# frank-medium — llama.cpp (DFlash2 drafter)
The ACTIVE frank-medium deployment since 2026-09-16 (model swapped to
The frank-medium deployment from 2026-09-16 to 2026-09-20 (model swapped to
orcarouter Q8_0 on 2026-09-19): **mainline llama.cpp**
serving `orcarouter/Qwen3.8-27B-Uncensored-GGUF` at `Q8_0`
with the **z-lab DFlash2** `Q4_K_M` block-diffusion drafter, as **one process across both
@@ -26,9 +31,11 @@ model (ajgazin Swift-Qwen3.8-27B UD-Q8_K_XL, 35:30 skew, 393216 pool) is the
calibration history in the sizing tables below. DSpark was tried in the drafter's
place on 2026-09-16 and reverted — see *The DSpark excursion*.
It replaced the vllm-radiance MXFP4 replica pair + routing proxy
([[frank-medium-radiance-mxfp4]], the record and the PCIe-atomics evidence) in
the same day the pair went live. It is also the box's **only** LLM runtime: the
replicas, the proxy and their unit files are retired.
([[frank-medium-radiance-mxfp4]] holds that record, and the PCIe-atomics
evidence) the same day the pair went live — and was itself replaced by
radiance TP=2 on 2026-09-20, once the board swap made a collective legal.
During its window it was the box's **only** LLM runtime: the replicas, the
proxy and their unit files were retired.
## Why one process and layer split