feat(monitoring,harness): follow frank-medium onto vllm:* metrics and fix red-by-construction targets
The radiance TP=2 cutover changed the metric namespace frank-medium exports (llamacpp:* -> vllm:*) and left two scrape targets addressing host processes by an address a container cannot reach. - shared/prometheus/rules/llm-fleet.yml: two namespaces side by side. Added LlmRequestsWaiting, LlmKvCacheSaturated (vllm-only gauges llama.cpp never had), the job:llm_vllm_prefix_cache_reuse:rate5m and job:llm_spec_accept_length:rate5m recording rules, and re-cut LlmPrefixCacheCollapse / LlmSpecDecodeOff to OR both families with per-family traffic floors so an idle endpoint cannot fire them. Every expression validated against live Prometheus. - shared/prometheus/prometheus.yml: frank-small -> host.containers.internal:7771 and frank-large scraped at host.containers.internal:7770. Both are HOST processes on Dankclaw and netavark DNATs published container ports only, so the LAN-IP targets were red by construction; frank-large's CIRU llama.cpp build does export the llamacpp:* family, contra the old comment. - tests/observability_quadlet.bats: the two seams now encode the new contract (per-endpoint reachability by process type; every named series must be on the measured list for its engine; engine-independent rules must name both). - homepage docs.html/services.json: frank-medium sees pictures, frank-large is CIRU Orca with a single seat (was halogen w4b, two seats, 200k). - docs: radiance runbook is ACTIVE (deployed flags, three-arm battery, verified capabilities, rollback, replica-era sections labelled); llama.cpp runbook carries the SUPERSEDED banner; Dual-R9700-Cooperation states the measured verdict; Observability records the namespace split and the DNAT lesson; AGENTS.md topology/frank-medium/Model Config/Benchmarking Policy/runbook index flipped; both indexes updated; work-log entry added. Also repaired agent-pass damage: prose prepended above frontmatter openers, duplicated runbook bodies, a literal tool-call token pasted into a runbook, and an index destroyed by diff artifacts (restored from HEAD, flips re-applied). Vault link check: 128 pages, 0 unresolved. ./tests/run.sh: 325 pass.
This commit is contained in:
1 parent
c2116658f8
commit
08f1c7ad09
13 files changed
+687
-286
No files matched your search
@@ -115,26 +115,22 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
|
||||
- [[deployment]] — unified deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[frank-medium-llamacpp-dflash2]] — the ACTIVE frank-medium runbook since
|
||||
2026-09-16 (model swapped 2026-09-19): mainline llama.cpp (gfx1201 HIP build)
|
||||
serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the DFlash2
|
||||
drafter (z-lab/Qwen3.8-27B-DFlash2-GGUF at Q4_K_M) on Dankserver's 2x Radeon
|
||||
AI PRO R9700 as one layer-split process behind the :7770 endpoint, balanced
|
||||
32:33 over two 163,840-token slots — no tensor parallelism: the runtime ships
|
||||
RCCL-less, and the collective has not been re-probed since the 2026-09-20
|
||||
board swap cleared the atomics gate
|
||||
- [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-16: the vllm-radiance
|
||||
MXFP4 stack serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq on
|
||||
Dankserver's 2x Radeon AI PRO R9700 as two single-GPU replicas behind a
|
||||
round-robin proxy; kept as the record of the deployment
|
||||
[[frank-medium-llamacpp-dflash2]] replaced, plus the PCIe-atomics evidence
|
||||
that ruled out TP=2 on the previous board (gate cleared 2026-09-20)
|
||||
- [[Dual-R9700-Cooperation]] — why the two R9700s *could not* run tensor-parallel
|
||||
together: the PCIe-atomics gate quoted from ROCclr, the measured RCCL matrix
|
||||
(collectives *and* P2P `send`/`recv` both failed on card #2, which ruled out
|
||||
pipeline parallelism too), what was closed, and the re-measured 2026-09-20
|
||||
topology after the ProArt B850-Creator x8/x8 swap — both chains `Routing+`,
|
||||
both endpoints `ReqEn+`, so TP=2 is unblocked but still unprobed
|
||||
- [[frank-medium-radiance-mxfp4]] — the ACTIVE frank-medium runbook since
|
||||
2026-09-20: vllm-radiance MXFP4 as ONE tensor-parallel process (TP=2 over the
|
||||
R4D one-shot allreduce) serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq
|
||||
with the z-lab DFlash2-FP8 drafter on Dankserver's 2x Radeon AI PRO R9700
|
||||
behind the :7770 endpoint — deployed flags, the three-arm battery that chose
|
||||
it, verified capabilities, and the record of the 2026-09-16 two-replica era
|
||||
- [[frank-medium-llamacpp-dflash2]] — RETIRED 2026-09-16 → 2026-09-20:
|
||||
mainline llama.cpp (gfx1201 HIP build) serving orcarouter Qwen3.8-27B at
|
||||
Q8_0 with the DFlash2 drafter as ONE layer-split process (32:33, two
|
||||
163,840-token slots); holds the batch/ubatch optimum curve and the
|
||||
per-layer split arithmetic the rollback recipe points back to
|
||||
- [[Dual-R9700-Cooperation]] — whether the two R9700s can run ONE model: the
|
||||
RCCL/PCIe-atomics evidence on the old TUF GAMING B650-E (which closed tensor
|
||||
parallelism too), what the 2026-09-20 ProArt B850-Creator x8/x8 swap changed
|
||||
— both chains `Routing+`, both endpoints `ReqEn+` — and the measured
|
||||
three-way verdict: radiance TP=2 won and is the deployed shape
|
||||
- [[frank-medium-rocmfpx]] — PRIOR frank-medium deployment (llama.cpp era):
|
||||
charlie12345/ROCmFPX serving kingjones777/
|
||||
Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF; kept as the record of the runtime it
|
||||
|
||||
Reference in new issue
Block a user