docs: dankserver board swap clears the PCIe-atomics wall
Reanalysis after the 2026-09-20 swap to an ASUS ProArt B850-Creator WiFi NEO: both R9700s now sit on CPU GPP root ports (x8/x8 Gen5 bifurcation), every bridge reads AtomicOpsCap Routing+, and both endpoints read AtomicOpsCtl ReqEn+ (CAP_EXP+28.w = 0x1440) — the exact precondition Dual-R9700-Cooperation's test plan named. TP=2 is therefore unblocked, not proven: no collective has been dispatched since the swap, GGML_HIP_RCCL is still OFF, and the 272 t/s "tensor" number on record measured a butterfly fallback. - Dual-R9700-Cooperation: short answer rewritten, re-measured 2026-09-20 topology table, rebuild-hypothesis door marked moot, probe plan is now standing work. - Dankserver System Specs: Motherboard section; GPU BDFs 0b:00.0 -> 07:00.0, iGPU 10:00.0 -> 11:00.0, kernel 7.0.0-31, SMT on, and the new eno2 (192.168.0.247) second 5GbE port holding the low-metric default route while eno1 keeps the fleet's static .235. - frank-medium-llamacpp-dflash2: rationale and hardware row split past from present; dated work-log sections stay as history. - AGENTS.md + both indexes: no longer assert a wall that no longer exists. Docs only; frank-medium stayed active and healthy throughout (verified /health). ./tests/run.sh 331/331.
This commit is contained in:
@@ -11,6 +11,40 @@ timestamp: 2026-09-14
|
||||
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-09-20 — dankserver board swap: the PCIe-atomics wall is gone
|
||||
- **Hardware.** dankserver now runs an **ASUS ProArt B850-Creator WiFi NEO**
|
||||
(was TUF GAMING B650-E WIFI), same Ryzen 7 7700X and same 2x R9700. The board
|
||||
bifurcates the CPU GPP ports **x8/x8 at Gen5**, so both cards sit on CPU root
|
||||
ports: card 0 `00:01.1 → 01:00.0 → 02:00.0 → 03:00.0`, card 1
|
||||
`00:01.3 → 05:00.0 → 06:00.0 → 07:00.0`. Every bridge reads
|
||||
`AtomicOpsCap: Routing+` and **both endpoints read `AtomicOpsCtl: ReqEn+`**
|
||||
(`CAP_EXP+28.w = 0x1440`) — the exact precondition
|
||||
[[Dual-R9700-Cooperation]]'s test plan named before any model would be
|
||||
touched. Bandwidth per card ~31.5 GB/s (kernel: `252.056 Gb/s … limited by
|
||||
32.0 GT/s PCIe x8`): card 0 unchanged from Gen4 x16, card 1 up ~16x from
|
||||
Gen4 x1.
|
||||
- **Verdict, stated honestly.** vLLM TP=2 / PP=2 and llama.cpp `--split-mode
|
||||
row` are **unblocked, not proven** — no collective has been dispatched since
|
||||
the swap, `GGML_HIP_RCCL` is still `OFF` in `scripts/build-llama-rdna4.sh`,
|
||||
and the 272 t/s "tensor" number on record measured a butterfly fallback, not
|
||||
an allreduce. `librccl.so.1` + `hipcc` already exist in the bundled TheRock
|
||||
toolchain, so an RCCL-backed rebuild needs no new host packages. The probe
|
||||
still owes a stop-and-mask window.
|
||||
- **Side effects of the swap, recorded.** GPU BDFs moved (`0b:00.0` → `07:00.0`),
|
||||
iGPU `10:00.0` → `11:00.0`, WiFi interface `wlp8s0` → `wlp14s0`, kernel now
|
||||
7.0.0-31-generic, SMT back **on** (16 logical CPUs; llama.cpp still `--threads
|
||||
8`), and the board's second 5GbE port is cabled as `eno2` = 192.168.0.247 with
|
||||
the lower-metric default route while `eno1` keeps the fleet's static
|
||||
192.168.0.235. Two addresses, two default routes — needs a decision.
|
||||
- **Docs cut over** (no service change; frank-medium stayed up and healthy
|
||||
throughout): [[Dual-R9700-Cooperation]] short answer rewritten plus a
|
||||
re-measured 2026-09-20 topology table; [[Dankserver System Specs]] gained a
|
||||
Motherboard section and corrected GPU/iGPU/network/kernel rows;
|
||||
[[frank-medium-llamacpp-dflash2]] hardware row and rationale now split past
|
||||
from present; `AGENTS.md` topology table, the frank-medium section, and the
|
||||
PCIe lesson bullets no longer assert a wall that no longer exists. The
|
||||
dated work-log sections that describe the old board stay as history.
|
||||
|
||||
## 2026-09-19 — fleet cutover: frank-large CIRU Orca, frank-medium orcarouter Q8_0, frank-small Gemma4 revert
|
||||
- **frank-large (Dankclaw :7770): halogen → CIRU v4.4.1 Orca.** The private
|
||||
CIRU v4.4.1 runtime (gfx1151 HIP build + QSA window hotfix, 381/381 ninja
|
||||
|
||||
Reference in New Issue
Block a user