diff --git a/scripts/frank-medium.env b/scripts/frank-medium.env index fbc64c5..a041c4a 100644 --- a/scripts/frank-medium.env +++ b/scripts/frank-medium.env @@ -77,6 +77,16 @@ # common_speculative_impl_draft_dflash::process). Handing three target layers # to ROCm0 buys back ~3 GiB of headroom on both cards at the full window. # +# EQUALITY WAS TRIED AND REJECTED. 39:26 levels the cards more closely (3.86 / +# 4.69 GiB free -- an 0.83 GiB gap, against 2.08 GiB here), but that server +# died on its eighth request: the amdgpu driver logged SDMA0 UTCL2 page faults +# and the engine surfaced `ROCm error: an illegal memory access was +# encountered` -> ggml_abort, i.e. a fault in the cross-card boundary copy, +# which is this layer split's hot path. 38:27 has since served the whole e2e +# gate plus a burst of varied requests with none. Do not chase an exact +# balance: take the skew that fits and leave the drafter's card on the safe +# side of the gap. +# # WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (HIP reads # those bare vars; 2026-08-13 incident). llama.cpp takes --host/--port/--temp # as flags only. @@ -103,8 +113,10 @@ DEVICES=ROCm0,ROCm1 SPLIT_MODE=layer # Skewed 38:27 rather than 1:1: ROCm1 also carries the drafter and the whole of -# its KV, so it must hold fewer target layers for the cards to end level. See -# SIZING above -- this is measured, not computed. +# its KV, so it must hold fewer target layers for the cards to end level -- at +# 1:1 the drafter's card would sit ~9 GiB above the other. This is the measured +# point that keeps BOTH cards above a 3 GiB floor at the full window; a tighter +# balance was tried and rejected (see SIZING above). TENSOR_SPLIT=38,27 NGL=999