diff --git a/scripts/frank-medium.env b/scripts/frank-medium.env index 6a38573..62d88c3 100644 --- a/scripts/frank-medium.env +++ b/scripts/frank-medium.env @@ -164,6 +164,26 @@ SPLIT_MODE=layer # 1.483 GiB slack) -- both gates fail. 35:30 (fit: 30.59 / 30.01, gap # 0.58) is the farthest safe shift toward the healthy card at this pool. # Do not move it without re-measuring. + +# MEASURED 2026-09-18 (isolated :7790, 2k-token decode, AVG rocm-smi util over +# the request -- peak util lies, a layer split touches both cards every token): +# +# split draft_dev decode AVG util0(x16) AVG util1(x1) +# 35,30 ROCm1 31.52 41 % 98 % <- deployed +# 36,29 ROCm1 31.49 43 % 98 % +# 35,30 ROCm0 31.52 42 % 98 % +# 36,29 ROCm0 31.48 43 % 98 % +# +# The x1 card stays the 98 % bottleneck in EVERY configuration. The lever that +# was tried is moving the MTP drafter to the x16 card (SPEC_DRAFT_DEVICE=ROCm0): +# it does NOT flip the balance, because the drafter is a small nextn head and the +# per-step cost is dominated by the VERIFY phase -- both cards run the full target +# -- plus the LOGITS, which are pinned to the card holding the last layers. A +# target-layer move (36:29) buys 2 points on the x16 card and erodes its VRAM +# margin (fit: free 0.45 GiB) for nothing. The x1 card is COMPUTE-bound, not +# PCIe-bound (~5.2 MB of activation per 512-token ubatch crosses the link); +# saturating it costs no tokens beyond what any R9700 would. 35:30 is the +# balance; do not chase the x16 card past it. TENSOR_SPLIT=35,30 NGL=999