docs: measured PP link math (H=5120 verified, 20.5 MB/ubatch wire), NIC inventory
- Ethernet Bandwidth Math: new section 0 with live measurements (iperf 281-294 MB/s, ping RTT both paths, prefill 457.9 t/s at 2.5GbE, per-ubatch wire 20.5 MB, 98k KV snapshot 3.45 GB blocking the slot 13.6 s, validated link-speed model: 386/458/505 t/s) - System specs: NIC tables for both boxes; the 1 GbE USB-C adapter on the claw was silently routing the RPC path (unplugged 2026-08-17) - PP Deployment: fix hidden-state size (10 KiB/direction, H=5120) and RTT-vs-bandwidth claim (decode RTT-bound; prefill+snapshot BW-bound)
This commit is contained in:
1 parent
968066ff23
commit
136ee080c9
4 files changed
+148
-7
No files matched your search
@@ -132,14 +132,21 @@ Dankclaw (head, 192.168.0.143) Dankserver (worker, 192.168.0.235)
|
||||
llama-server b10433 ggml-rpc-server b10433
|
||||
--rpc 192.168.0.235:50052 --host 192.168.0.235 --port 50052
|
||||
model file (31.5 GB) -d CUDA0 -c (tensor cache)
|
||||
[RPC0, ROCm0|Vulkan0] layers RTX 3090: layers 0..~31
|
||||
Strix: layers ~32..64 + lm_head
|
||||
hidden state ~20 KiB crosses the link once per token, each direction
|
||||
[RPC0, ROCm0|Vulkan0] layers RTX 3090: layers 0-44 (45 of 65 blocks)
|
||||
Strix: layers 45-64 + lm_head
|
||||
hidden state 10 KiB/position/direction (H=5120, GGUF-verified);
|
||||
measured wire bytes ~40 KB/position/direction (serialized graph
|
||||
dominates); 20.5 MB per ubatch both directions. Details:
|
||||
[[Ethernet Bandwidth Math]] section 0.
|
||||
```
|
||||
|
||||
- **Layer-sequential split** (true pipeline parallelism): layers 0..N run on
|
||||
the worker, the rest on the head, in order. One hidden-state handoff per
|
||||
token step. RTT-bound, not bandwidth-bound.
|
||||
token step. DECODE is RTT-bound, not bandwidth-bound (network is ~1-2.5%
|
||||
of step time). PREFILL and the prompt-cache KV snapshot ARE
|
||||
bandwidth-bound: the link runs at line rate for the wire burst of every
|
||||
ubatch and ~100% during the snapshot copy. Measured numbers:
|
||||
[[Ethernet Bandwidth Math]] section 0.
|
||||
- **Why the head is on Dankclaw:** the model file is already there; the
|
||||
worker streams its tensor share at load (no 31.5 GB copy to the server).
|
||||
- **Both sides must run the identical build** (b10433 / 9b05354ec). The
|
||||
|
||||
Reference in new issue
Block a user