docs: dankserver board swap clears the PCIe-atomics wall

Reanalysis after the 2026-09-20 swap to an ASUS ProArt B850-Creator WiFi
NEO: both R9700s now sit on CPU GPP root ports (x8/x8 Gen5 bifurcation),
every bridge reads AtomicOpsCap Routing+, and both endpoints read
AtomicOpsCtl ReqEn+ (CAP_EXP+28.w = 0x1440) — the exact precondition
Dual-R9700-Cooperation's test plan named. TP=2 is therefore unblocked,
not proven: no collective has been dispatched since the swap,
GGML_HIP_RCCL is still OFF, and the 272 t/s "tensor" number on record
measured a butterfly fallback.

- Dual-R9700-Cooperation: short answer rewritten, re-measured 2026-09-20
  topology table, rebuild-hypothesis door marked moot, probe plan is now
  standing work.
- Dankserver System Specs: Motherboard section; GPU BDFs 0b:00.0 ->
  07:00.0, iGPU 10:00.0 -> 11:00.0, kernel 7.0.0-31, SMT on, and the new
  eno2 (192.168.0.247) second 5GbE port holding the low-metric default
  route while eno1 keeps the fleet's static .235.
- frank-medium-llamacpp-dflash2: rationale and hardware row split past
  from present; dated work-log sections stay as history.
- AGENTS.md + both indexes: no longer assert a wall that no longer exists.

Docs only; frank-medium stayed active and healthy throughout (verified
/health). ./tests/run.sh 331/331.
This commit is contained in:
Frank
2026-09-20 14:39:54 -07:00
parent a43ab90105
commit d16e8e554e
7 changed files with 246 additions and 110 deletions
+34
View File
@@ -11,6 +11,40 @@ timestamp: 2026-09-14
# Bundle Change Log
## 2026-09-20 — dankserver board swap: the PCIe-atomics wall is gone
- **Hardware.** dankserver now runs an **ASUS ProArt B850-Creator WiFi NEO**
(was TUF GAMING B650-E WIFI), same Ryzen 7 7700X and same 2x R9700. The board
bifurcates the CPU GPP ports **x8/x8 at Gen5**, so both cards sit on CPU root
ports: card 0 `00:01.1 → 01:00.0 → 02:00.0 → 03:00.0`, card 1
`00:01.3 → 05:00.0 → 06:00.0 → 07:00.0`. Every bridge reads
`AtomicOpsCap: Routing+` and **both endpoints read `AtomicOpsCtl: ReqEn+`**
(`CAP_EXP+28.w = 0x1440`) — the exact precondition
[[Dual-R9700-Cooperation]]'s test plan named before any model would be
touched. Bandwidth per card ~31.5 GB/s (kernel: `252.056 Gb/s … limited by
32.0 GT/s PCIe x8`): card 0 unchanged from Gen4 x16, card 1 up ~16x from
Gen4 x1.
- **Verdict, stated honestly.** vLLM TP=2 / PP=2 and llama.cpp `--split-mode
row` are **unblocked, not proven** — no collective has been dispatched since
the swap, `GGML_HIP_RCCL` is still `OFF` in `scripts/build-llama-rdna4.sh`,
and the 272 t/s "tensor" number on record measured a butterfly fallback, not
an allreduce. `librccl.so.1` + `hipcc` already exist in the bundled TheRock
toolchain, so an RCCL-backed rebuild needs no new host packages. The probe
still owes a stop-and-mask window.
- **Side effects of the swap, recorded.** GPU BDFs moved (`0b:00.0` → `07:00.0`),
iGPU `10:00.0` → `11:00.0`, WiFi interface `wlp8s0` → `wlp14s0`, kernel now
7.0.0-31-generic, SMT back **on** (16 logical CPUs; llama.cpp still `--threads
8`), and the board's second 5GbE port is cabled as `eno2` = 192.168.0.247 with
the lower-metric default route while `eno1` keeps the fleet's static
192.168.0.235. Two addresses, two default routes — needs a decision.
- **Docs cut over** (no service change; frank-medium stayed up and healthy
throughout): [[Dual-R9700-Cooperation]] short answer rewritten plus a
re-measured 2026-09-20 topology table; [[Dankserver System Specs]] gained a
Motherboard section and corrected GPU/iGPU/network/kernel rows;
[[frank-medium-llamacpp-dflash2]] hardware row and rationale now split past
from present; `AGENTS.md` topology table, the frank-medium section, and the
PCIe lesson bullets no longer assert a wall that no longer exists. The
dated work-log sections that describe the old board stay as history.
## 2026-09-19 — fleet cutover: frank-large CIRU Orca, frank-medium orcarouter Q8_0, frank-small Gemma4 revert
- **frank-large (Dankclaw :7770): halogen → CIRU v4.4.1 Orca.** The private
CIRU v4.4.1 runtime (gfx1151 HIP build + QSA window hotfix, 381/381 ninja