frank-medium: shift the layer split to 35:30, leaning work onto the x16 card
The two R9700s sit on unequal PCIe paths: ROCm0 (03:00.0) is CPU-root-port Gen4 x16 (~31.5 GB/s); ROCm1 (0b:00.0) rides the B650 chipset through bridges negotiating Gen4 x1 (~1.97 GB/s). With the MTP drafter, output layers and heavier compute buffers pinned to the last-layers card, live rocm-smi read 98% busy on the x1 card vs 40% on the x16 card. Moving one DeltaNet layer (34) to ROCm0 shifts ~0.43 GiB off the bottlenecked card; the seam fit lands 30.59/30.01 with gap 0.58 GiB, inside floor and slack. 36:29 fails both gates, so this lever is spent.
This commit is contained in:
1 parent
d6bc245934
commit
6569c63ec4
4 files changed
+83
-25
No files matched your search
@@ -158,7 +158,9 @@ print('draft-model wiring ok')
|
||||
# measured 6.44 / 6.90 GiB free. The fit reproduces card 1 to 0.00 GiB
|
||||
# and card 0 to 0.86 GiB at that point; it is a GATE, not a meter -- the
|
||||
# deployed point is re-measured on the host after every split/ctx/batch
|
||||
# move (the current one is 34:31 / 393216 / MTP, 2026-09-17).
|
||||
# move (the current one is 35:30 / 393216 / MTP, moved from 34:31 on
|
||||
# 2026-09-17 to lean work off ROCm1's Gen4-x1 chipset path -- see the
|
||||
# TENSOR_SPLIT header in frank-medium.env).
|
||||
#
|
||||
# card 0 = its share of the weights and of the KV (qwen35 spends KV on
|
||||
# its 16 full-attention layers only: 2 (K,V) x 4 heads x 256 dim
|
||||
@@ -196,19 +198,20 @@ print('draft-model wiring ok')
|
||||
local kv_pos=65536 draft_kv_pos=20480 mtp_kv_pos=4096 floor slack
|
||||
local card_hi card_lo diff
|
||||
local mmproj=931145888 # mmproj-BF16.gguf, autoloaded onto the primary device
|
||||
# 1.5 GiB was the floor until the 2x196k shape: the deployed 34:31 / 393216
|
||||
# / MTP point measures 28.91 / 30.78 GiB, leaving 2.95 / 1.08 GiB, so the
|
||||
# 1.5 GiB was the floor until the 2x196k shape: the 34:31 / 393216 /
|
||||
# MTP point measured 28.91 / 30.78 GiB, leaving 2.95 / 1.08 GiB, so the
|
||||
# floor is the measured 1.08 GiB rounded down. The margin for one big
|
||||
# allocation is thinner than it used to be; if a large allocation fails at
|
||||
# load, the first move is a layer to ROCm0 (33:32), not a ctx cut.
|
||||
# allocation is thinner than it used to be. The layer-toward-ROCm0
|
||||
# lever has since been spent (35:30, 2026-09-17): if a large allocation
|
||||
# fails at load now, the next lever is a ctx cut, not another layer.
|
||||
floor=$(( 1024 * 1024 * 1024 )) # 1.0 GiB per card
|
||||
# The deployed point (34:31 at ctx 393216 with MTP, the balance point at
|
||||
# this pool) has a post-recalibration fit gap of ~1.0 GiB; the slack sits
|
||||
# above it at 1.483 GiB, which still rejects a one-layer move (worth
|
||||
# ~0.43-1.43 GiB of gap depending on the boundary) while accepting the
|
||||
# deployed split. (The 36:29 / 327680 / DFlash2 configuration's slack was
|
||||
# 3225 MiB -- its 3.15 GiB gap was by construction, and is no longer the
|
||||
# deployed shape.)
|
||||
# The deployed point (35:30 at ctx 393216 with MTP) has a fit gap of
|
||||
# 0.58 GiB -- closer to balanced than its 34:31 predecessor (1.06 GiB).
|
||||
# The slack sits at 1.483 GiB: it accepts the deployed split and still
|
||||
# rejects the next step toward ROCm0, 36:29, which fails BOTH gates
|
||||
# (gap 2.22 GiB, card 0 free 0.45 GiB under the floor). (The 36:29 /
|
||||
# 327680 / DFlash2 configuration's slack was 3225 MiB -- its 3.15 GiB
|
||||
# gap was by construction, and is no longer the deployed shape.)
|
||||
slack=$(( 1483 * 1024 * 1024 ))
|
||||
card_hi=$(( MODEL_BYTES * split_hi / (split_hi + split_lo) \
|
||||
+ kv_pos * ctx * split_hi / (split_hi + split_lo) \
|
||||
|
||||
Reference in new issue
Block a user