docs: measured PP link math (H=5120 verified, 20.5 MB/ubatch wire), NIC inventory

- Ethernet Bandwidth Math: new section 0 with live measurements
  (iperf 281-294 MB/s, ping RTT both paths, prefill 457.9 t/s at
  2.5GbE, per-ubatch wire 20.5 MB, 98k KV snapshot 3.45 GB blocking
  the slot 13.6 s, validated link-speed model: 386/458/505 t/s)
- System specs: NIC tables for both boxes; the 1 GbE USB-C adapter on
  the claw was silently routing the RPC path (unplugged 2026-08-17)
- PP Deployment: fix hidden-state size (10 KiB/direction, H=5120) and
  RTT-vs-bandwidth claim (decode RTT-bound; prefill+snapshot BW-bound)
This commit is contained in:
Frank committed 2026-08-17 05:06:41 -07:00
1 parent 968066ff23
commit 136ee080c9
4 files changed
+148 -7

No files matched your search

@@ -132,14 +132,21 @@ Dankclaw (head, 192.168.0.143) Dankserver (worker, 192.168.0.235)
llama-server b10433 ggml-rpc-server b10433
--rpc 192.168.0.235:50052 --host 192.168.0.235 --port 50052
model file (31.5 GB) -d CUDA0 -c (tensor cache)
[RPC0, ROCm0|Vulkan0] layers RTX 3090: layers 0..~31
Strix: layers ~32..64 + lm_head
hidden state ~20 KiB crosses the link once per token, each direction
[RPC0, ROCm0|Vulkan0] layers RTX 3090: layers 0-44 (45 of 65 blocks)
Strix: layers 45-64 + lm_head
hidden state 10 KiB/position/direction (H=5120, GGUF-verified);
measured wire bytes ~40 KB/position/direction (serialized graph
dominates); 20.5 MB per ubatch both directions. Details:
[[Ethernet Bandwidth Math]] section 0.
```
- **Layer-sequential split** (true pipeline parallelism): layers 0..N run on
the worker, the rest on the head, in order. One hidden-state handoff per
token step. RTT-bound, not bandwidth-bound.
token step. DECODE is RTT-bound, not bandwidth-bound (network is ~1-2.5%
of step time). PREFILL and the prompt-cache KV snapshot ARE
bandwidth-bound: the link runs at line rate for the wire burst of every
ubatch and ~100% during the snapshot copy. Measured numbers:
[[Ethernet Bandwidth Math]] section 0.
- **Why the head is on Dankclaw:** the model file is already there; the
worker streams its tensor share at load (no 31.5 GB copy to the server).
- **Both sides must run the identical build** (b10433 / 9b05354ec). The