the x4 link, measured: 6.72 GB/s — which decides the streaming question

pcie-link-test.cu (CUDA, pinned buffers, cudaEvent timing, both services stopped):
pinned H2D 6.72 GB/s and D2H 6.60 GB/s, i.e. 85 % of the Gen4 x4 ceiling. Async
is no better than sync and pageable sits within 1 % of pinned, so these copies are
link-limited rather than staging-limited — the usual "pageable costs 2x" does not
apply on this host. A 3 MiB slab costs 474 us, essentially 3 MiB / 6.72 GB/s plus
~27 us of launch.

That settles the expert-streaming question quantitatively: 1.70 GiB of active
experts per token (A3B @ Q4_K_M, 75 GiB expert mass) is 272 ms of link time ⇒
3.7 tok/s, 4.6 with a fifth of the experts resident. The same bytes computed in
host RAM at 256 GB/s cost 7.1 ms — the link is 31x slower than the memory behind
it, which is why llama.cpp's -ot exps=CPU path measured 20.46 tok/s while an
engine that streams loses by 3-5x. The port under the card is x8-capable and the
card is x16-capable, so the x4 is the adapter; doubling it still leaves streaming
well behind.

Also records FreeToken's own constraint: it keeps experts in host RAM with an LRU
GPU cache (exactly this streaming shape) and loads only modelopt block-FP8
safetensors for qwen3_5_moe, so Q4_K_M GGUF is unloadable there regardless.

AGENTS.md

@@ -415,6 +415,24 @@ linked runbooks.
   target that works, because the CPU backend is last in the list. Pure Vulkan
   changes nothing here — the gate is the override, not the driver mix. See
   [[x4-pair-experiment]].
+- **A host↔3090 link at x4 measures 6.72 GB/s, and that is a design
+  constraint, not a detail.** `pcie-link-test.cu` on Dankclaw (2026-09-16,
+  services stopped): pinned H2D 6.72 GB/s, D2H 6.60 GB/s — 85 % of the Gen4 x4
+  ceiling — with async no better than sync and pageable within 1 % of pinned,
+  i.e. the copies are link-limited rather than staging-limited (the usual
+  "pageable costs 2x" rule does not apply here). The card is x16-capable and the
+  Strix Halo port beneath it is x8-capable (`LnkCap: 16GT/s, Width x8` on
+  `00:03.1`), so the x4 lives in the adapter or cable; the link idles at
+  2.5 GT/s x4 and trains up under load. Consequence for any engine that
+  *streams* weights across it: a 3 MiB slab costs 474 us, and a 111 GiB A3B
+  model needs ~1.70 GiB of active experts per token ⇒ **3.7 tok/s** (4.6 with a
+  20 % resident cache), against **20.46 tok/s** for the same model with the
+  experts *computed* in host RAM by llama.cpp (`-ot exps=CPU`, whence the link
+  carries activations only) and 25–26.5 on the iGPU. Streaming over PCIe loses;
+  computing in RAM does not care. FreeToken — expert LRU in host RAM, exactly
+  this streaming shape — also loads only modelopt block-FP8 safetensors for
+  `qwen3_5_moe`, so a Q4_K_M GGUF is unloadable there before bandwidth is even
+  the question. See [[x4-pair-experiment]].
 
 
 **ROCm / GPU**