frank-medium: step 3 -- the skew is the balance, and equality is off-limits
The 38:27 skew folded into the DSpark swap already does what step 3 asks: the draft-carrying card sits 2.08 GiB above the other (5.49 / 3.41 GiB free) instead of the ~9 GiB a 1:1 split would leave. Chasing an exact balance is a mistake worth recording: 39:26 measured tighter (3.86 / 4.69 GiB free) but that server died on its eighth request with SDMA0 UTCL2 page faults in the amdgpu driver -- "ROCm error: an illegal memory access was encountered" -> ggml_abort, in the cross-card boundary copy that is the layer split's hot path. 38:27 has since served the whole e2e gate and 12 varied requests with no fault, so the residual 2.08 GiB gap is the safe side of this trade. Comment-only: no restart needed.
This commit is contained in:
@@ -77,6 +77,16 @@
|
||||
# common_speculative_impl_draft_dflash::process). Handing three target layers
|
||||
# to ROCm0 buys back ~3 GiB of headroom on both cards at the full window.
|
||||
#
|
||||
# EQUALITY WAS TRIED AND REJECTED. 39:26 levels the cards more closely (3.86 /
|
||||
# 4.69 GiB free -- an 0.83 GiB gap, against 2.08 GiB here), but that server
|
||||
# died on its eighth request: the amdgpu driver logged SDMA0 UTCL2 page faults
|
||||
# and the engine surfaced `ROCm error: an illegal memory access was
|
||||
# encountered` -> ggml_abort, i.e. a fault in the cross-card boundary copy,
|
||||
# which is this layer split's hot path. 38:27 has since served the whole e2e
|
||||
# gate plus a burst of varied requests with none. Do not chase an exact
|
||||
# balance: take the skew that fits and leave the drafter's card on the safe
|
||||
# side of the gap.
|
||||
#
|
||||
# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (HIP reads
|
||||
# those bare vars; 2026-08-13 incident). llama.cpp takes --host/--port/--temp
|
||||
# as flags only.
|
||||
@@ -103,8 +113,10 @@ DEVICES=ROCm0,ROCm1
|
||||
SPLIT_MODE=layer
|
||||
|
||||
# Skewed 38:27 rather than 1:1: ROCm1 also carries the drafter and the whole of
|
||||
# its KV, so it must hold fewer target layers for the cards to end level. See
|
||||
# SIZING above -- this is measured, not computed.
|
||||
# its KV, so it must hold fewer target layers for the cards to end level -- at
|
||||
# 1:1 the drafter's card would sit ~9 GiB above the other. This is the measured
|
||||
# point that keeps BOTH cards above a 3 GiB floor at the full window; a tighter
|
||||
# balance was tried and rejected (see SIZING above).
|
||||
TENSOR_SPLIT=38,27
|
||||
|
||||
NGL=999
|
||||
|
||||
Reference in New Issue
Block a user