feat(monitoring,harness): follow frank-medium onto vllm:* metrics and fix red-by-construction targets
The radiance TP=2 cutover changed the metric namespace frank-medium exports (llamacpp:* -> vllm:*) and left two scrape targets addressing host processes by an address a container cannot reach. - shared/prometheus/rules/llm-fleet.yml: two namespaces side by side. Added LlmRequestsWaiting, LlmKvCacheSaturated (vllm-only gauges llama.cpp never had), the job:llm_vllm_prefix_cache_reuse:rate5m and job:llm_spec_accept_length:rate5m recording rules, and re-cut LlmPrefixCacheCollapse / LlmSpecDecodeOff to OR both families with per-family traffic floors so an idle endpoint cannot fire them. Every expression validated against live Prometheus. - shared/prometheus/prometheus.yml: frank-small -> host.containers.internal:7771 and frank-large scraped at host.containers.internal:7770. Both are HOST processes on Dankclaw and netavark DNATs published container ports only, so the LAN-IP targets were red by construction; frank-large's CIRU llama.cpp build does export the llamacpp:* family, contra the old comment. - tests/observability_quadlet.bats: the two seams now encode the new contract (per-endpoint reachability by process type; every named series must be on the measured list for its engine; engine-independent rules must name both). - homepage docs.html/services.json: frank-medium sees pictures, frank-large is CIRU Orca with a single seat (was halogen w4b, two seats, 200k). - docs: radiance runbook is ACTIVE (deployed flags, three-arm battery, verified capabilities, rollback, replica-era sections labelled); llama.cpp runbook carries the SUPERSEDED banner; Dual-R9700-Cooperation states the measured verdict; Observability records the namespace split and the DNAT lesson; AGENTS.md topology/frank-medium/Model Config/Benchmarking Policy/runbook index flipped; both indexes updated; work-log entry added. Also repaired agent-pass damage: prose prepended above frontmatter openers, duplicated runbook bodies, a literal tool-call token pasted into a runbook, and an index destroyed by diff artifacts (restored from HEAD, flips re-applied). Vault link check: 128 pages, 0 unresolved. ./tests/run.sh: 325 pass.
This commit is contained in:
1 parent
c2116658f8
commit
08f1c7ad09
13 files changed
+687
-286
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: configuration
|
||||
title: frank-medium llama.cpp Runbook (DFlash2 drafter)
|
||||
description: The ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
|
||||
description: The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
|
||||
tags:
|
||||
- frank-medium
|
||||
- dankserver
|
||||
@@ -12,12 +12,17 @@ tags:
|
||||
- gfx1201
|
||||
- r9700
|
||||
- layer-split
|
||||
timestamp: 2026-09-19
|
||||
timestamp: 2026-09-20
|
||||
---
|
||||
|
||||
> **SUPERSEDED 2026-09-20.** Replaced by [[frank-medium-radiance-mxfp4]]
|
||||
> (radiance TP=2, measured faster on every axis). The layer-split shape below is
|
||||
> the rollback target, and the llama.cpp measurements in this page stay valid for
|
||||
> it — but nothing here is the deployed shape any more.
|
||||
|
||||
# frank-medium — llama.cpp (DFlash2 drafter)
|
||||
|
||||
The ACTIVE frank-medium deployment since 2026-09-16 (model swapped to
|
||||
The frank-medium deployment from 2026-09-16 to 2026-09-20 (model swapped to
|
||||
orcarouter Q8_0 on 2026-09-19): **mainline llama.cpp**
|
||||
serving `orcarouter/Qwen3.8-27B-Uncensored-GGUF` at `Q8_0`
|
||||
with the **z-lab DFlash2** `Q4_K_M` block-diffusion drafter, as **one process across both
|
||||
@@ -26,9 +31,11 @@ model (ajgazin Swift-Qwen3.8-27B UD-Q8_K_XL, 35:30 skew, 393216 pool) is the
|
||||
calibration history in the sizing tables below. DSpark was tried in the drafter's
|
||||
place on 2026-09-16 and reverted — see *The DSpark excursion*.
|
||||
It replaced the vllm-radiance MXFP4 replica pair + routing proxy
|
||||
([[frank-medium-radiance-mxfp4]], the record and the PCIe-atomics evidence) in
|
||||
the same day the pair went live. It is also the box's **only** LLM runtime: the
|
||||
replicas, the proxy and their unit files are retired.
|
||||
([[frank-medium-radiance-mxfp4]] holds that record, and the PCIe-atomics
|
||||
evidence) the same day the pair went live — and was itself replaced by
|
||||
radiance TP=2 on 2026-09-20, once the board swap made a collective legal.
|
||||
During its window it was the box's **only** LLM runtime: the replicas, the
|
||||
proxy and their unit files were retired.
|
||||
|
||||
## Why one process and layer split
|
||||
|
||||
|
||||
Reference in new issue
Block a user