From ad18d2ddd8fb5147a6decdb0d46dff6977889cc7 Mon Sep 17 00:00:00 2001 From: Frank Date: Wed, 16 Sep 2026 20:00:17 -0700 Subject: [PATCH] frank-medium: step 3 -- the skew is the balance, and equality is off-limits The 38:27 skew folded into the DSpark swap already does what step 3 asks: the draft-carrying card sits 2.08 GiB above the other (5.49 / 3.41 GiB free) instead of the ~9 GiB a 1:1 split would leave. Chasing an exact balance is a mistake worth recording: 39:26 measured tighter (3.86 / 4.69 GiB free) but that server died on its eighth request with SDMA0 UTCL2 page faults in the amdgpu driver -- "ROCm error: an illegal memory access was encountered" -> ggml_abort, in the cross-card boundary copy that is the layer split's hot path. 38:27 has since served the whole e2e gate and 12 varied requests with no fault, so the residual 2.08 GiB gap is the safe side of this trade. Comment-only: no restart needed. --- scripts/frank-medium.env | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/scripts/frank-medium.env b/scripts/frank-medium.env index fbc64c5..a041c4a 100644 --- a/scripts/frank-medium.env +++ b/scripts/frank-medium.env @@ -77,6 +77,16 @@ # common_speculative_impl_draft_dflash::process). Handing three target layers # to ROCm0 buys back ~3 GiB of headroom on both cards at the full window. # +# EQUALITY WAS TRIED AND REJECTED. 39:26 levels the cards more closely (3.86 / +# 4.69 GiB free -- an 0.83 GiB gap, against 2.08 GiB here), but that server +# died on its eighth request: the amdgpu driver logged SDMA0 UTCL2 page faults +# and the engine surfaced `ROCm error: an illegal memory access was +# encountered` -> ggml_abort, i.e. a fault in the cross-card boundary copy, +# which is this layer split's hot path. 38:27 has since served the whole e2e +# gate plus a burst of varied requests with none. Do not chase an exact +# balance: take the skew that fits and leave the drafter's card on the safe +# side of the gap. +# # WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (HIP reads # those bare vars; 2026-08-13 incident). llama.cpp takes --host/--port/--temp # as flags only. @@ -103,8 +113,10 @@ DEVICES=ROCm0,ROCm1 SPLIT_MODE=layer # Skewed 38:27 rather than 1:1: ROCm1 also carries the drafter and the whole of -# its KV, so it must hold fewer target layers for the cards to end level. See -# SIZING above -- this is measured, not computed. +# its KV, so it must hold fewer target layers for the cards to end level -- at +# 1:1 the drafter's card would sit ~9 GiB above the other. This is the measured +# point that keeps BOTH cards above a 3 GiB floor at the full window; a tighter +# balance was tried and rejected (see SIZING above). TENSOR_SPLIT=38,27 NGL=999