frank-medium: seam recalibrated to measured MTP charge; docs to MTP/393216/34:31

The 393216/34:31/MTP deploy measured 30.78 GiB on ROCm1 against the
fit's 28.32 — the MTP draft buffers (one nextn layer's KV, 4096 B/pos,
plus draft buffers) came in 2.46 GiB above the estimate. Recalibrate
the seam's MTP constant to the measured 4.20 GiB total charge on card
1 and drop the floor 1.5 -> 1.0 GiB for this shape (ROCm1's 1.08 GiB
headroom is real; 35:30/36:29 fail card 0's floor at this pool, so
34:31 is the only split that fits 2x196k).

AGENTS.md and the runbook move to the MTP drafter: 34:31, 393216 pool
(196608 per conversation), the DFlash2 sidecar documented as a
one-line revert, and a dated part-2 section with the fit/measured
table and the recalibration.
This commit is contained in:
Frank committed 2026-09-17 03:35:46 -07:00
1 parent 4d13333da7
commit 74dda30fc7
3 files changed
+98 -42

No files matched your search

+17 -7
View File
@@ -196,10 +196,17 @@ print('draft-model wiring ok')
local kv_pos=65536 draft_kv_pos=20480 mtp_kv_pos=4096 floor slack
local card_hi card_lo diff
local mmproj=931145888 # mmproj-BF16.gguf, autoloaded onto the primary device
floor=$(( 1536 * 1024 * 1024 )) # 1.5 GiB per card
# The current point is 34:31 at ctx 393216 with MTP: the fit gap is 1.44
# GiB, which is also the balance point at this pool, so the slack is that
# gap rounded up. (The 36:29 / 327680 / DFlash2 configuration's slack was
# 1.5 GiB was the floor until the 2x196k shape: the deployed 34:31 / 393216
# / MTP point measures 28.91 / 30.78 GiB, leaving 2.95 / 1.08 GiB, so the
# floor is the measured 1.08 GiB rounded down. The margin for one big
# allocation is thinner than it used to be; if a large allocation fails at
# load, the first move is a layer to ROCm0 (33:32), not a ctx cut.
floor=$(( 1024 * 1024 * 1024 )) # 1.0 GiB per card
# The deployed point (34:31 at ctx 393216 with MTP, the balance point at
# this pool) has a post-recalibration fit gap of ~1.0 GiB; the slack sits
# above it at 1.483 GiB, which still rejects a one-layer move (worth
# ~0.43-1.43 GiB of gap depending on the boundary) while accepting the
# deployed split. (The 36:29 / 327680 / DFlash2 configuration's slack was
# 3225 MiB -- its 3.15 GiB gap was by construction, and is no longer the
# deployed shape.)
slack=$(( 1483 * 1024 * 1024 ))
@@ -217,12 +224,15 @@ print('draft-model wiring ok')
+ 2468 * 1024 * ubatch ))
else
# MTP: the head is in the target file (charged above); one nextn
# layer's KV plus ~0.2 GiB of buffers (estimate -- re-measure on the
# host after a drafter swap, as always).
# layer's KV plus buffers/graphs. The constant is CALIBRATED to the
# deployed point (2026-09-17, 34:31 / 393216): card 1 measured 30.78
# GiB, of which 4.20 GiB is MTP's total charge -- 1.61 GiB of nextn KV
# plus 2.70 GiB of buffers and graphs, far more than the 0.2 GiB
# estimate the DFlash2-era shape suggested.
card_lo=$(( MODEL_BYTES * split_lo / (split_hi + split_lo) \
+ kv_pos * ctx * split_lo / (split_hi + split_lo) \
+ mtp_kv_pos * ctx \
+ 209715200 \
+ 2901231913 \
+ 2468 * 1024 * ubatch ))
fi
[ "$card_hi" -le $(( CARD_BYTES - floor )) ]