frank-medium: shift the layer split to 35:30, leaning work onto the x16 card

The two R9700s sit on unequal PCIe paths: ROCm0 (03:00.0) is CPU-root-port
Gen4 x16 (~31.5 GB/s); ROCm1 (0b:00.0) rides the B650 chipset through
bridges negotiating Gen4 x1 (~1.97 GB/s).  With the MTP drafter, output
layers and heavier compute buffers pinned to the last-layers card, live
rocm-smi read 98% busy on the x1 card vs 40% on the x16 card.  Moving one
DeltaNet layer (34) to ROCm0 shifts ~0.43 GiB off the bottlenecked card;
the seam fit lands 30.59/30.01 with gap 0.58 GiB, inside floor and slack.
36:29 fails both gates, so this lever is spent.
This commit is contained in:
Frank committed 2026-09-17 18:58:06 -07:00
1 parent d6bc245934
commit 6569c63ec4
4 files changed
+83 -25

No files matched your search

+15 -12
View File
@@ -158,7 +158,9 @@ print('draft-model wiring ok')
# measured 6.44 / 6.90 GiB free. The fit reproduces card 1 to 0.00 GiB
# and card 0 to 0.86 GiB at that point; it is a GATE, not a meter -- the
# deployed point is re-measured on the host after every split/ctx/batch
# move (the current one is 34:31 / 393216 / MTP, 2026-09-17).
# move (the current one is 35:30 / 393216 / MTP, moved from 34:31 on
# 2026-09-17 to lean work off ROCm1's Gen4-x1 chipset path -- see the
# TENSOR_SPLIT header in frank-medium.env).
#
# card 0 = its share of the weights and of the KV (qwen35 spends KV on
# its 16 full-attention layers only: 2 (K,V) x 4 heads x 256 dim
@@ -196,19 +198,20 @@ print('draft-model wiring ok')
local kv_pos=65536 draft_kv_pos=20480 mtp_kv_pos=4096 floor slack
local card_hi card_lo diff
local mmproj=931145888 # mmproj-BF16.gguf, autoloaded onto the primary device
# 1.5 GiB was the floor until the 2x196k shape: the deployed 34:31 / 393216
# / MTP point measures 28.91 / 30.78 GiB, leaving 2.95 / 1.08 GiB, so the
# 1.5 GiB was the floor until the 2x196k shape: the 34:31 / 393216 /
# MTP point measured 28.91 / 30.78 GiB, leaving 2.95 / 1.08 GiB, so the
# floor is the measured 1.08 GiB rounded down. The margin for one big
# allocation is thinner than it used to be; if a large allocation fails at
# load, the first move is a layer to ROCm0 (33:32), not a ctx cut.
# allocation is thinner than it used to be. The layer-toward-ROCm0
# lever has since been spent (35:30, 2026-09-17): if a large allocation
# fails at load now, the next lever is a ctx cut, not another layer.
floor=$(( 1024 * 1024 * 1024 )) # 1.0 GiB per card
# The deployed point (34:31 at ctx 393216 with MTP, the balance point at
# this pool) has a post-recalibration fit gap of ~1.0 GiB; the slack sits
# above it at 1.483 GiB, which still rejects a one-layer move (worth
# ~0.43-1.43 GiB of gap depending on the boundary) while accepting the
# deployed split. (The 36:29 / 327680 / DFlash2 configuration's slack was
# 3225 MiB -- its 3.15 GiB gap was by construction, and is no longer the
# deployed shape.)
# The deployed point (35:30 at ctx 393216 with MTP) has a fit gap of
# 0.58 GiB -- closer to balanced than its 34:31 predecessor (1.06 GiB).
# The slack sits at 1.483 GiB: it accepts the deployed split and still
# rejects the next step toward ROCm0, 36:29, which fails BOTH gates
# (gap 2.22 GiB, card 0 free 0.45 GiB under the floor). (The 36:29 /
# 327680 / DFlash2 configuration's slack was 3225 MiB -- its 3.15 GiB
# gap was by construction, and is no longer the deployed shape.)
slack=$(( 1483 * 1024 * 1024 ))
card_hi=$(( MODEL_BYTES * split_hi / (split_hi + split_lo) \
+ kv_pos * ctx * split_hi / (split_hi + split_lo) \