docs(vault): fleet-wide fact re-verification pass (67 pages)

The docs-fact census wave re-read the live hosts and re-keyed every page
it could: runtime/payload identities, ports, sizes, ledger figures, unit
state, and the retirement banners that drifted as the fleet moved. The
wave was interrupted mid-run, so this commit carries only what passed the
integrity gates: every touched page parses (front matter or H1+status
convention intact, balanced fences, no truncated tables, trailing
newline), the docs_completeness and link-integrity seams are green, and
bats is 384/384.

Numbers that this session proved on live hosts are keyed to the measured
value, not the page's earlier claim.

serverAppConfigs-docs/LLM/Dual-R9700-Cooperation.md

@@ -14,11 +14,15 @@ timestamp: 2026-09-16
 
 # Dual R9700 Cooperation
 
-Dankserver's two R9700s serve `frank-medium` as **one vLLM-radiance process at
-tensor_parallel_size 2** ([[frank-medium-radiance-mxfp4]]). This page records why
+Dankserver's two R9700s run **one tensor-parallel engine across both cards** —
+today that is `frank-archon` under radiance 1.0.13, TP=2
+([[frank-archon-radiance-qwen38-iq4r]]). When this page was written the endpoint
+was `frank-medium`: first llama.cpp's layer split
+([[frank-medium-llamacpp-dflash2]], 2026-09-16 → 09-20), then from 2026-09-20
+one vLLM-radiance process at `tensor_parallel_size 2`
+([[frank-medium-radiance-mxfp4]], 2026-09-20 → 09-26). This page records why
 true tensor parallelism was impossible on the previous board (TUF GAMING B650-E
-WIFI), and what the 2026-09-20 board swap changed. The llama.cpp layer split that
-served the endpoint until that swap is [[frank-medium-llamacpp-dflash2]].
+WIFI), and what the 2026-09-20 board swap changed.
 
 ## The short answer
 
@@ -43,9 +47,11 @@ What that bought:
   and served (its R4D allreduce needs no RCCL collective at all), and llama.cpp
   `--split-mode tensor` booted and served against an RCCL-backed build — so a
   collective HAS been dispatched on this host since the swap.
-- **One of them is production.** TP=2 wins on every axis measured, by 2.1x on
-  decode and 3.6x at 4-way concurrency, so it replaced the layer split on the day
-  of measurement. See [[frank-medium-radiance-mxfp4]] for the deployed shape.
+- **TP=2 is the production shape.** It wins on every axis measured, by 2.1x on
+  decode and 3.6x at 4-way concurrency, and it replaced the layer split on the day
+  of measurement. The payload has moved since: the MXFP4 `frank-medium` shape is
+  retired 2026-09-26, and what runs today is the iq4r publisher under radiance
+  1.0.13 ([[frank-archon-radiance-qwen38-iq4r]]).
 - **The drafter, not the collective, is what llama.cpp lost.** Its tensor mode
   could not carry the DFlash2 drafter, which is the honest caveat on the
   comparison below.
@@ -137,20 +143,30 @@ LAN/SATA/WiFi:
 - `dmesg` agrees on bandwidth: `252.056 Gb/s available PCIe bandwidth, limited
   by 32.0 GT/s PCIe x8 link at 0000:00:01.1` (and `.13`) — ~31.5 GB/s per
   card. DMI confirms the bifurcation: `PCIEX16(G5)_2`, "8x or x8".
-- `rocm-smi --showtopo` still reports `PCIE` between the cards (Navi 48 has no
-  XGMI), they are still in separate IOMMU groups (15 and 20), and both root
-  ports have ACS `ReqRedir+`, so P2P hairpins at the root complex. Plain P2P
-  traffic measured 45.27 GB/s bidirectional on the *old* board — DMA was never
-  the gate, atomics were.
-- Kernel 7.0.0-31-generic with `iommu=pt`. Both were ruled out as causes
-  ([above](#ruled-out-empirically-do-not-re-derive)) and both are unchanged.
+- `rocm-smi --showtopo` reported `PCIE` between the cards (Navi 48 has no XGMI),
+  they were in separate IOMMU groups (15 and 20 — re-read 2026-10-18: still 15
+  and 20), and both root ports had ACS `ReqRedir+`, so this page inferred P2P
+  would hairpin at the root complex. **That inference is superseded:** the live
+  engine says the opposite about the same pair — "I device: the 2 cards reach
+  each other's memory directly (peer to peer)" (`podman logs frank-archon`,
+  2026-10-15 boot). Plain P2P traffic measured 45.27 GB/s bidirectional on the
+  *old* board — DMA was never the gate, atomics were.
+- Kernel 7.0.0-31-generic with `iommu=pt` then; this host runs 7.0.0-38-generic
+  with `iommu=pt` today (`/proc/cmdline`, read 2026-10-18). Both were ruled out
+  as causes ([above](#ruled-out-empirically-do-not-re-derive)) and this host's
+  IOMMU posture is unchanged.
 
-**Still unproven.** The gate that matters is ROCclr's `checkAtomicSupport()`
-reading the `kLinkAtomicSupport` link attribute; the ReqEn bit is the same cause
-seen from the kernel side, but no RCCL kernel has been launched since the swap.
-Until the 2-rank probe runs — or a `--split-mode row` boot that does *not* log
-`internal AllReduce init failed ... falling back to meta-backend butterfly` —
-"collectives work here" is a strong expectation, not a measurement.
+**Still unproven (as of 2026-09-20).** The gate that matters is ROCclr's
+`checkAtomicSupport()` reading the `kLinkAtomicSupport` link attribute; the ReqEn
+bit is the same cause seen from the kernel side, but no RCCL kernel had been
+launched since the swap. Until the 2-rank probe runs — or a `--split-mode row`
+boot that does *not* log `internal AllReduce init failed ... falling back to
+meta-backend butterfly` — "collectives work here" was a strong expectation, not a
+measurement. **RCCL never got that measurement**; what settled the operational
+question instead is production traffic: radiance's TP=2 wire moves cross-rank
+all-reduces over the direct P2P path and reports its own precision — "tp wire:
+LOSSY (--tp-wire wht6) at 128 KiB a message and above … Steps of at most 25 rows
+are exact" (`podman logs frank-archon`, 2026-10-15 boot).
 
 ## Measured matrix
 
@@ -249,9 +265,13 @@ RCCL 2.27 does carry gfx1201 kernels, and that is what row 3 linked against.
 
 **Rule for any future multi-GPU collective experiment: stop and mask the
 production service first**, per the benchmarking policy, or run it on a host that
-is not serving users. Since 2026-09-20 `frank-medium` is a rootless quadlet, so
-that is `systemctl --user stop frank-medium && systemctl --user mask frank-medium`
-on Dankserver.
+is not serving users. The production service is `frank-archon`, a **root** unit,
+so on Dankserver that is `sudo systemctl stop frank-archon && sudo systemctl
+mask frank-archon` (`scripts/frank-archon.service`; never `disable` — that
+deletes the unit file, and `mask` is what the bench policy prescribes). The
+2026-09-20 wording below — `systemctl --user stop frank-medium && systemctl
+--user mask frank-medium`, frank-medium being a rootless quadlet — is retired
+with that service, which today reports `not-found` (read 2026-10-18).
 
 This is not theoretical: on 2026-09-16 a bare RCCL P2P probe was run against the live
 service; the host went hard down ~20 minutes later (no ARP, no ICMP, required a
@@ -260,7 +280,7 @@ GPU driver is a plausible cause and this host has precedent for going
 unreachable until a power cycle.
 
 
-## The third path, and the one in production: llama.cpp's layer split
+## The third path (production in the layer-split era): llama.cpp's layer split
 
 Everything above is about *collectives*, and vLLM cannot proceed without them
 in any of its parallelisms — TP needs a per-layer allreduce, and PP's stage
@@ -270,8 +290,9 @@ layers are distributed across the cards and the boundary activation is a
 plain per-backend copy, once per forward pass. No RCCL kernel is ever launched
 on the second card, so the atomics gate never comes up.
 
-That is what `frank-medium` runs today ([[frank-medium-llamacpp-dflash2]]): one
-process, one model, both cards, 262144 context. Measured against tensor split
+That is what `frank-medium` ran from 2026-09-16 to 09-20
+([[frank-medium-llamacpp-dflash2]]): one process, one model, both cards,
+262144 context. Measured against tensor split
 on this host: 1991 t/s prefill / 18.8 tok/s decode layer-split, against 272 /
 26.3 tensor (butterfly fallback, no RCCL). **That tensor number is not a
 prediction for an RCCL-backed build** — it is what tensor split costs when the
@@ -282,9 +303,10 @@ The ranking therefore changed twice. On 2026-09-16: software already gets both
 cards onto one model, so the hardware fix stopped being a capability question
 and became a performance one — *tensor* parallelism, ~40 % decode, plus one
 pooled KV instead of two independent 122k-token pools. On 2026-09-20 the
-hardware half of that question was answered by the board swap; the measurement
-half has not run yet, and `GGML_HIP_RCCL` is still `OFF` in
-`scripts/build-llama-rdna4.sh` for exactly the reason this page documents.
+hardware half of that question was answered by the board swap; the llama.cpp
+measurement half never ran, and `GGML_HIP_RCCL` still defaults `OFF` in
+`scripts/build-llama-rdna4.sh:41` (re-read 2026-10-18) for exactly the reason
+this page documents — moot while no llama.cpp shape is deployed on this host.
 
 ## What would change the answer