The operator's revised spec: batch x3 (6144/1536), parallel 2, a 1.5 GiB per-card
floor, and the context maximised inside it.
Measured at 262144 with the old 38:27 split, 3x batch was brutally asymmetric:
ROCm1 (output layers + the drafter's graphs) paid ~3.3 GiB where ROCm0 paid
~1.7, leaving the draft-carrying card at 0.08 GiB free against ROCm0's 3.78.
One -ts step is worth ~1.5 GiB at this batch, so the balance is a measurement:
41:24 overshot (1.47 / 2.38), 40:25 landed at 1.92 / 1.96 GiB free.
That balance is also what maximises the context -- an unbalanced split would have
to buy its headroom out of CTX_SIZE, and CTX_SIZE is already at the model's
trained window (262144), now split over two slots of 131072. Verified under two
concurrent 1832-token prompts: 200/200, 1.87 / 1.86 GiB free, n_slots = 2,
n_ctx_slot = 131072, tool calls intact, and DSpark drafting on BOTH slots
(acceptance 0.38/0.22 and 0.52/0.43).
8192/2048 does not fit at this context at all: its 2020 MiB pp buffer fails to
allocate on ROCm1, and 4096/1024 fits but leaves 1.11 GiB -- both recorded in
the env's sizing table.
The unit seam now models both cards (share + draft + ubatch-scaled buffers,
calibrated to the rocm-smi figures to within 0.002%), asserts the 1.5 GiB floor
on each, and asserts the two cards stay within 0.5 GiB of each other. Both
assertions were confirmed to FAIL on a deliberately bad config (ubatch 8192, and
a 1:1 split) rather than pass by construction.
Frank
b312042706