docs: pipeline parallelism research + educational sample script set

Compiles the 5-subagent research fan-out (llama.cpp RPC, vLLM PP,
bandwidth math, mixed-vendor experiments, Qwen3.6-27B spec methods) into
the KB under docs/Pipeline Parallelism/ with an index, expected
performance estimates (1/2.5/10 GbE), and an EDUCATIONAL-ONLY two-node
Qwen3.6-27B Q8 split script set (MTP/DFlash/ngram-mod variants, DSpark
noted as vLLM-only). Nothing here is deployed or intended to run.

docs/Pipeline Parallelism/Expected Performance.md

@@ -0,0 +1,97 @@
+---
+type: reference
+title: Expected Performance
+description: Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors.
+tags:
+  - pipeline-parallelism
+  - performance
+  - estimate
+timestamp: 2026-08-14
+---
+
+# Expected Performance
+
+This page estimates the speed of a 25-40B model split across two machines
+over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]]
+and the measured numbers in [[Mixed-Vendor Experiments]].
+
+## The Bandwidth Ceiling Is Not the Limit
+
+A pipeline boundary moves only the hidden state. For a 27B model (hidden
+5120), that is 10240 bytes per token per boundary in fp16. Even at 64
+tokens per second with batch 64, the traffic is about 262 Mbit/s. 1 GbE can
+carry it. The real limits are latency, RPC overhead, and the slowest node.
+
+The Petals paper measured this directly. Dropping from 1 Gbit/s to
+100 Mbit/s changed throughput by 3 percent. Raising round-trip latency from
+5 ms to 100 ms dropped throughput by 28 percent. Latency matters more than
+bandwidth for pipeline inference.
+
+## Real Benchmark Anchors
+
+| Experiment | Model | Network | Decode (t/s) |
+| ---------- | ----- | ------- | ------------ |
+| Mac Studio + DGX Spark (Metal+CUDA) | Qwen2.5-72B Q4 | 10 GbE | 6 |
+| Mac Studio + DGX Spark (Metal+CUDA) | Qwen2.5-7B Q4 | 10 GbE | 53 |
+| RTX 4070 + MI50 + 3050 (CUDA+ROCm) | Qwen3-VL-30B Q5 | 1 GbE | 51.5 clean, 17.8 at 32k ctx |
+| Two Strix Halos (ROCm RPC) | MiniMax-M2.5-139B Q8 | USB4 | 15.35 |
+| AMD four-node Strix Halo (5 GbE) | Kimi K2.5 (1T) | 5 GbE | 8.30 |
+| Tesla V100 + R9700x2 (Vulkan, one box) | Qwen3.6-35B-A3B Q8 | PCIe | 62 |
+
+Pattern: RPC decode runs about 2-5 times slower than the best single node.
+Prefill benefits from more compute across nodes. The 30B-class model over
+1 GbE with warm cache reached 51.5 t/s on a short context and 17.8 t/s at
+32k context.
+
+## Estimate for Qwen3.6-27B Q8, Two Nodes
+
+Scenario: RTX 3090 (24 GB VRAM, NVIDIA desktop) plus Strix Halo NUC
+(Radeon 8060S, 30 GB physical RAM). The model is the dense 27B at Q8
+(29-32 GB). Each node holds part of the weights and KV cache.
+
+| Link | Decode (t/s) | Prefill (t/s) | Notes |
+| ---- | ------------ | ------------- | ----- |
+| 1 GbE | 8-15 | 200-500 | Latency-bound; per-token RPC round trip |
+| 2.5 GbE | 10-18 | 250-600 | Some headroom; latency still dominant |
+| 10 GbE | 12-20 | 300-800 | Closest to the published 10 GbE runs |
+
+The dense 27B decodes slower per token than the MoE 30B A3B class. Expect
+the lower half of the ranges at full context.
+
+### With Speculative Decoding
+
+Speculative decoding (MTP, DFlash, or ngram-mod) raises effective decode.
+Tightwad measured 1.86x on a slow RPC pool. Estimate the effective rate at
+1.4-1.8x the raw RPC decode:
+
+| Link | Raw decode (t/s) | With spec (t/s) |
+| ---- | ---------------- | --------------- |
+| 1 GbE | 8-15 | 12-24 |
+| 2.5 GbE | 10-18 | 15-28 |
+| 10 GbE | 12-20 | 18-32 |
+
+## Context That Fits
+
+The two real machines give about 51 GiB of usable memory combined
+(24 GiB VRAM plus about 27 GiB on the Strix Halo, which has 30 GiB physical
+RAM; the 96 GiB driver ceiling is not physical). A 29-32 GB model leaves
+about 20-22 GiB for KV cache. The Qwen3.6-27B KV cache costs about 85-100
+KiB per token with q8_0 K and f16 V (measured on the large service at
+602100 tokens = about 50 GB). That budget gives a total context pool of
+roughly 200k-260k tokens across both nodes.
+
+## Caveats
+
+- Every node must run the exact same llama.cpp build. Version skew hangs
+  the handshake.
+- The slowest node sets the pace for every token.
+- Model load crosses the network once per start. Expect minutes, not
+  seconds, for the split load.
+- These are estimates. The exact combination of an RTX 3090 desktop and a
+  Strix Halo NUC has no published benchmark yet.
+
+## See Also
+
+- [[Ethernet Bandwidth Math]]
+- [[Mixed-Vendor Experiments]]
+- [[Sample Script Set]]