frank-medium: seam recalibrated to measured MTP charge; docs to MTP/393216/34:31
The 393216/34:31/MTP deploy measured 30.78 GiB on ROCm1 against the fit's 28.32 — the MTP draft buffers (one nextn layer's KV, 4096 B/pos, plus draft buffers) came in 2.46 GiB above the estimate. Recalibrate the seam's MTP constant to the measured 4.20 GiB total charge on card 1 and drop the floor 1.5 -> 1.0 GiB for this shape (ROCm1's 1.08 GiB headroom is real; 35:30/36:29 fail card 0's floor at this pool, so 34:31 is the only split that fits 2x196k). AGENTS.md and the runbook move to the MTP drafter: 34:31, 393216 pool (196608 per conversation), the DFlash2 sidecar documented as a one-line revert, and a dated part-2 section with the fit/measured table and the recalibration.
This commit is contained in:
1 parent
4d13333da7
commit
74dda30fc7
3 files changed
+98
-42
No files matched your search
@@ -196,10 +196,17 @@ print('draft-model wiring ok')
|
||||
local kv_pos=65536 draft_kv_pos=20480 mtp_kv_pos=4096 floor slack
|
||||
local card_hi card_lo diff
|
||||
local mmproj=931145888 # mmproj-BF16.gguf, autoloaded onto the primary device
|
||||
floor=$(( 1536 * 1024 * 1024 )) # 1.5 GiB per card
|
||||
# The current point is 34:31 at ctx 393216 with MTP: the fit gap is 1.44
|
||||
# GiB, which is also the balance point at this pool, so the slack is that
|
||||
# gap rounded up. (The 36:29 / 327680 / DFlash2 configuration's slack was
|
||||
# 1.5 GiB was the floor until the 2x196k shape: the deployed 34:31 / 393216
|
||||
# / MTP point measures 28.91 / 30.78 GiB, leaving 2.95 / 1.08 GiB, so the
|
||||
# floor is the measured 1.08 GiB rounded down. The margin for one big
|
||||
# allocation is thinner than it used to be; if a large allocation fails at
|
||||
# load, the first move is a layer to ROCm0 (33:32), not a ctx cut.
|
||||
floor=$(( 1024 * 1024 * 1024 )) # 1.0 GiB per card
|
||||
# The deployed point (34:31 at ctx 393216 with MTP, the balance point at
|
||||
# this pool) has a post-recalibration fit gap of ~1.0 GiB; the slack sits
|
||||
# above it at 1.483 GiB, which still rejects a one-layer move (worth
|
||||
# ~0.43-1.43 GiB of gap depending on the boundary) while accepting the
|
||||
# deployed split. (The 36:29 / 327680 / DFlash2 configuration's slack was
|
||||
# 3225 MiB -- its 3.15 GiB gap was by construction, and is no longer the
|
||||
# deployed shape.)
|
||||
slack=$(( 1483 * 1024 * 1024 ))
|
||||
@@ -217,12 +224,15 @@ print('draft-model wiring ok')
|
||||
+ 2468 * 1024 * ubatch ))
|
||||
else
|
||||
# MTP: the head is in the target file (charged above); one nextn
|
||||
# layer's KV plus ~0.2 GiB of buffers (estimate -- re-measure on the
|
||||
# host after a drafter swap, as always).
|
||||
# layer's KV plus buffers/graphs. The constant is CALIBRATED to the
|
||||
# deployed point (2026-09-17, 34:31 / 393216): card 1 measured 30.78
|
||||
# GiB, of which 4.20 GiB is MTP's total charge -- 1.61 GiB of nextn KV
|
||||
# plus 2.70 GiB of buffers and graphs, far more than the 0.2 GiB
|
||||
# estimate the DFlash2-era shape suggested.
|
||||
card_lo=$(( MODEL_BYTES * split_lo / (split_hi + split_lo) \
|
||||
+ kv_pos * ctx * split_lo / (split_hi + split_lo) \
|
||||
+ mtp_kv_pos * ctx \
|
||||
+ 209715200 \
|
||||
+ 2901231913 \
|
||||
+ 2468 * 1024 * ubatch ))
|
||||
fi
|
||||
[ "$card_hi" -le $(( CARD_BYTES - floor )) ]
|
||||
|
||||
Reference in new issue
Block a user