Isolated-port (:7790) 2k-token decode with AVG rocm-smi util over the
request (peak util lies: a layer split touches both cards every token):
35:30 draft ROCm1 31.52 t/s util0 41% util1 98% <- deployed
36:29 draft ROCm1 31.49 t/s util0 43% util1 98%
35:30 draft ROCm0 31.52 t/s util0 42% util1 98%
36:29 draft ROCm0 31.48 t/s util0 43% util1 98%
The x1 card (ROCm1) stays the 98% bottleneck in every configuration.
Moving the MTP drafter to the x16 card does not flip the balance: the
per-step cost is the verify phase (both cards run the full target) plus
the logits, which are pinned to the card holding the last layers. A
target-layer move (36:29) buys 2 points on the x16 card and erodes its
VRAM margin (fit free 0.45 GiB) for no throughput change. The x1 card is
compute-bound, not PCIe-bound (~5.2 MB activation per ubatch on the link),
so saturating it costs no tokens. 35:30 is the balance; no split change.
Frank
6c6464a31a