dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs(vault): strip restated fleet facts from 17 technique pages

Browse Source 
Enforces "one fact, one owner": technique pages keep mechanism, laws, traps
and verdicts; per-endpoint shapes, digests and era measurements are cited to
the endpoint page that owns them.

Also repairs the four pointers the pass left empty (each verified against the
cited owner before landing):
- EMULATION dequant doubles host RAM (118 GiB -> ~238 GiB) -> [[frank-archon]]
- 12.11 tok/s flat 304 -> 16,095 tokens -> [[Tcclaviger Fork Ops]]
- bundled-head acceptance ladder 0.739/0.554/0.427/0.340 -> [[Qwen3.8 PP Deployment]]
- llama.cpp PR #23897 dedup provenance -> [[qwen3.8-spec-matrix]]

Replaces the retired-shape illustration in Llama Cpp Serving Sizing with the
owned MiMo Q5_K_M row (6.40 GiB payload -> 20,530 MiB used), and unwraps a
hard-wrapped [[Dankserver System Specs]] link that failed link integrity.

Orphan census over the diff: 0 numbers lost (comma-insensitive match against
the whole vault plus AGENTS.md).
This commit is contained in:
Frank
2026-10-19 01:33:47 -07:00
parent eb21dbc705
commit 04f4e2c6d4
18 changed files with 348 additions and 425 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/LLM/Dual-R9700-Cooperation.md
+28 -31
View File
@@ -147,10 +147,10 @@ LAN/SATA/WiFi:
they were in separate IOMMU groups (15 and 20 — re-read 2026-10-18: still 15
and 20), and both root ports had ACS `ReqRedir+`, so this page inferred P2P
would hairpin at the root complex. **That inference is superseded:** the live
engine says the opposite about the same pair — "I device: the 2 cards reach
each other's memory directly (peer to peer)" (`podman logs frank-archon`,
2026-10-15 boot). Plain P2P traffic measured 45.27 GB/s bidirectional on the
*old* board — DMA was never the gate, atomics were.
engine reports the two cards reaching each other's memory directly, which
[[frank-archon]] and [[Dankserver System Specs]] record. Plain P2P traffic
measured 45.27 GB/s bidirectional on the *old* board — DMA was never the gate,
atomics were.
- Kernel 7.0.0-31-generic with `iommu=pt` then; this host runs 7.0.0-38-generic
with `iommu=pt` today (`/proc/cmdline`, read 2026-10-18). Both were ruled out
as causes ([above](#ruled-out-empirically-do-not-re-derive)) and this host's
@@ -163,10 +163,9 @@ launched since the swap. Until the 2-rank probe runs — or a `--split-mode row`
boot that does *not* log `internal AllReduce init failed ... falling back to
meta-backend butterfly` — "collectives work here" was a strong expectation, not a
measurement. **RCCL never got that measurement**; what settled the operational
question instead is production traffic: radiance's TP=2 wire moves cross-rank
all-reduces over the direct P2P path and reports its own precision — "tp wire:
LOSSY (--tp-wire wht6) at 128 KiB a message and above … Steps of at most 25 rows
are exact" (`podman logs frank-archon`, 2026-10-15 boot).
question instead is production traffic — radiance's TP=2 wire carries cross-rank
all-reduces over the direct P2P path and prints its own precision, and the
measured `wht6` banner with its exact-step bound is on [[frank-archon]].
## Measured matrix
@@ -233,35 +232,34 @@ landed, at x8/x8, clears the arithmetic with room to spare.
Both cooperation modes were run on this host on 2026-09-20 — the same salted
10-prompt battery (`scripts/bench-conversation.py`), the service stopped and
masked between arms. Decode/prefill are geometric means over the ten rows;
burst-4 is four concurrent copies of one prompt, reported as aggregate tokens/s.
masked between arms. The arm table is [[frank-medium-radiance-mxfp4]]'s bench of
record (arms R = radiance TP=2 + DFlash2 as deployed, I = the llama.cpp layer
split that preceded it, L = llama.cpp `--split-mode tensor` against RCCL); this
page keeps only the verdict. **TP=2 won on every axis — roughly 2.1× decode and
3.6× at 4-way concurrency over the layer split — and replaced the layer split the
day of measurement.**
| Shape | Cross-device path | Decode | TTFT | Prefill | burst-4 |
|---|---|---|---|---|---|
| **radiance TP=2 + DFlash2 — deployed** | R4D one-shot allreduce | **95.8 t/s** | **0.21 s** | **1493.6 t/s** | **259.4 t/s (65.6/stream)** |
| llama.cpp layer split 32:33 + DFlash2 — predecessor | none, boundary activation copy | 45.7 t/s | 0.72 s | ~530 t/s | 71.7 t/s |
| llama.cpp `--split-mode tensor` 1:1 | RCCL allreduce | 30.5 t/s | 0.71 s | n/r | 54.3 t/s |
Row 3 carries a caveat that no flag can remove: llama.cpp's tensor mode could not
load the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
Arm L carries a caveat no flag can remove: llama.cpp's tensor mode could not load
the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
pre-allocated on the non-primary device has no backend once `n_copies == 1`), so
it decoded **without** speculation while the other two rows decoded with it. The
comparison that decides anything is row 1 against row 2, and some of that gap is
the 4-bit MXFP4 checkpoint rather than the parallelism.
it decoded **without** speculation while the other arms decoded with it; and part
of R's margin over I is the 4-bit MXFP4 checkpoint, not the parallelism.
Two numbers elsewhere on this page are therefore not interchangeable:
- The 272 t/s recorded 2026-09-16 is **prefill**, on a **butterfly** allreduce,
with RCCL off, on the old board. It was never a TP decode number.
- 26.3 t/s from the same table is that butterfly tensor's decode. Row 3's 30.5 is
an RCCL tensor decode a day later: same shape family, different collective path.
with RCCL off, on the old board. It was never a TP decode number, and its
paired decode figure is in "The third path" below.
- The butterfly tensor arm and the RCCL tensor arm a day later are the same shape
family on different collective paths; the measured pair lives with
[[frank-medium-radiance-mxfp4]]'s arms.
RCCL needs PCIe atomics on *every* participating GPU — the gate the old board
failed. vLLM's R4D one-shot allreduce dispatches no collective at all, which is
why TP=2 runs today rather than merely "being unblocked": the image ships TheRock
RCCL, and TheRock carries no gfx1201 device code (`invalid device function`), so
radiance could not have used RCCL on this host even with atomics routed. Ubuntu's
RCCL 2.27 does carry gfx1201 kernels, and that is what row 3 linked against.
RCCL 2.27 does carry gfx1201 kernels, and that is what arm L linked against.
**Rule for any future multi-GPU collective experiment: stop and mask the
production service first**, per the benchmarking policy, or run it on a host that
@@ -292,12 +290,11 @@ on the second card, so the atomics gate never comes up.
That is what `frank-medium` ran from 2026-09-16 to 09-20
([[frank-medium-llamacpp-dflash2]]): one process, one model, both cards,
262144 context. Measured against tensor split
on this host: 1991 t/s prefill / 18.8 tok/s decode layer-split, against 272 /
26.3 tensor (butterfly fallback, no RCCL). **That tensor number is not a
prediction for an RCCL-backed build** — it is what tensor split costs when the
allreduce degrades to a meta-backend butterfly, and it is the number the
board-swap path exists to retire.
262144 context. Its measured layer-split rates are that page's; this host's
tensor arm under the butterfly fallback measured 272 t/s prefill / 26.3 tok/s
decode. **That tensor number is not a prediction for an RCCL-backed build** — it
is what tensor split costs when the allreduce degrades to a meta-backend
butterfly, and it is the number the board-swap path exists to retire.
The ranking therefore changed twice. On 2026-09-16: software already gets both
cards onto one model, so the hardware fix stopped being a capability question