docs(vault): fleet-wide fact re-verification pass (67 pages)

The docs-fact census wave re-read the live hosts and re-keyed every page
it could: runtime/payload identities, ports, sizes, ledger figures, unit
state, and the retirement banners that drifted as the fleet moved. The
wave was interrupted mid-run, so this commit carries only what passed the
integrity gates: every touched page parses (front matter or H1+status
convention intact, balanced fences, no truncated tables, trailing
newline), the docs_completeness and link-integrity seams are green, and
bats is 384/384.

Numbers that this session proved on live hosts are keyed to the measured
value, not the page's earlier claim.
This commit is contained in:
Frank committed 2026-10-18 13:15:43 -07:00
1 parent 466fd477d8
commit e7f0d93c86
67 files changed
+2619 -1152

No files matched your search

@@ -59,9 +59,11 @@ Gen4 usable payload after 128b/130b: **x4 7.877 GB/s, x8 15.75, x16 31.5**.
**Ceiling arithmetic:** `link time/token = active expert bytes / measured
H2D rate`; `ceiling = 1 / link_time × resident-fraction saved (LRU hits)`.
Compare against the compute-in-RAM path at host RAM bandwidth (256 GB/s
LPDDR5X on the Strix Halo). **If the link is >10x slower than the memory
behind it (x4 is 31x here), streaming loses by construction and no tuning
rescues it; doubling the link only doubles a losing number.**
LPDDR5X on the Strix Halo — 128 GB LPDDR5x-8000 on a 256-bit bus,
[amd.com spec page](https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html)).
**If the link is >10x slower than the memory behind it (x4 is 31x here),
streaming loses by construction and no tuning rescues it; doubling the link
only doubles a losing number.**
Worked example (Dankclaw 2026-09-16): Gen4 x4 measured H2D **6.72 / D2H 6.60
GB/s** (85 % of spec) — pinned H2D ≈ async, pageable within 1 %, i.e.
@@ -215,15 +217,25 @@ reporting an outage (`gitea` vs `gitea-server`).
complex; x4 suffices for TP.
2. Board whose CPU x16 bifurcates x8/x8 (done 2026-09-20 on dankserver — both
R9700s on root ports `00:01.1`/`00:01.3`, every bridge `Routing+`, both
endpoints `ReqEn+`).
endpoints `ReqEn+`). Re-checked 2026-10-18 without root: both root ports
report `current_link_width=8` at `32.0 GT/s`
(`/sys/bus/pci/devices/0000:00:01.{1,3}/current_link_width`), and the live
engine states direct P2P between the cards
([[Dual-R9700-Cooperation]]).
3. llama.cpp `--split-mode layer` — no collectives at all ([[Llama Cpp Multi-Device Splits]]); costs the GGUF conversion plus paged attention /
continuous batching / spec decode.
4. Independent TP=1 replicas + routing proxy — cards never share one model.
**Closed software routes (do not re-litigate):** vLLM's IPC custom allreduce
is gated by a CUDA-sm-keyed table (`{9.0, 10.0, 10.3, 10.7}`; ROCm reports
12.0 → off on RDNA4 by code limit) — and patching it is insufficient because
TP init's `broadcast`/`barrier`/logits `all_gather` have no non-RCCL path. A
12.0 → off on RDNA4 by code limit) — the table is
`CUSTOM_ALL_REDUCE_MAX_SIZES` in
`vllm/distributed/device_communicators/all_reduce_utils.py`, whose keys on
today's `main` are exactly `"9.0"`, `"10.0"`, `"10.3"`, `"10.7"`
([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/distributed/device_communicators/all_reduce_utils.py),
read 2026-10-18); nothing keys on a ROCm-reported `12.0`. — and patching it is
insufficient because TP init's `broadcast`/`barrier`/logits `all_gather` have
no non-RCCL path. A
fork wrapping `.all_reduce` (e.g. radiance `RADIANCE_USE_R4D_AR` one-shot
P2P, gloo for IPC-handle exchange) does not intercept
`PyNcclCommunicator.__init__`'s warmup allreduce. NCCL symmetric memory needs
@@ -241,4 +253,4 @@ kernel's declared args (`roc-obj-extract` + `llvm-readelf --notes`), look for
- [[Llama Cpp Multi-Device Splits]] — the layer-split path around the wall
- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — measured history
- [[dankserver-qwen38-fp8-expert-offload-tuning]] — the live gather economics
- [[dankserver-qwen38-fp8-expert-offload-tuning]] — the gather economics of the retired tcclaviger vLLM era (2026-09-29 → 10-05)