fix(run-swift15-bf16): kv pool 3072 both offline arms
The weights slab derives as card - headroom - kv, and the disk tier refuses under 9.57 GiB (4.21 layer buffers + 5.36 fixed). 4608 starved it (8.68 budget, DID NOT FIT). 3072 leaves 10.2 for weights; 4x16k fp8 slots fit the pool, recording is prefill-bound so slot count only queues pushes.
This commit is contained in:
1 parent
4048dc8630
commit
9aed9987b6
1 file changed
+13
-7
@@ -46,23 +46,29 @@ ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad
|
||||
case "$MODE" in
|
||||
kld)
|
||||
# 1.0.13: the engine that will serve the candidate — the reference is
|
||||
# recorded on the same engine that scores it. The ratio-derived KV pool
|
||||
# (1.95 GiB) is SHORT by 217 MiB at 32x16k; the pair is explicit: 16 slots
|
||||
# need 2.17 fixed + 1.87 GiB of fp8 KV, 4608 MiB carries the margin.
|
||||
# recorded on the same engine that scores it. The weights slab DERIVES as
|
||||
# card - headroom - kv, and the disk tier refuses under 9.57 GiB (4.21 of
|
||||
# layer buffers + 5.36 fixed), so kv is explicit SMALL: recording is
|
||||
# prefill-only, self-batched at MNBT 32768 = two 16k prompts at a time;
|
||||
# 4 slots x 16k fp8 (29 KiB/pos, 1.7 GiB) + headroom fits 3072.
|
||||
IMAGE="stilldeadcode/radiance@$PIN_1013"
|
||||
mkdir -p "$KLD_REF_DIR"
|
||||
ARGS+=(--kld-record /data/kld-ref --kld-corpus /data/kld.jsonl
|
||||
--vram-kv-mib 4608
|
||||
--max-num-seqs 16 --max-num-batched-tokens 32768)
|
||||
--vram-kv-mib 3072
|
||||
--max-num-seqs 4 --max-num-batched-tokens 32768)
|
||||
;;
|
||||
calib)
|
||||
# 1.2.0: the calibration tap (RADIANCE_CALIB_DIR + RADIANCE_CALIB_TOKENS)
|
||||
# lives in the 1.2.0 qwen4exp plugin; 1.0.13 has only the DIR half with an
|
||||
# unparameterisable threshold. Grams are bf16 activations — engine-invariant.
|
||||
# Same KV ceiling as kld: the disk-tier weights floor (9.57 GiB) plus
|
||||
# ~18 GiB of fixed per-rank claims cap the pool near 3.5 GiB on a 32 GiB
|
||||
# card. 4 slots x 16k fp8 fits; prefill rate is disk-bound, so fewer
|
||||
# resident slots only queues pushes, it does not slow the record.
|
||||
IMAGE="stilldeadcode/radiance@$PIN_120"
|
||||
mkdir -p "$CALIB_DIR"
|
||||
ARGS+=(--port 7770 --vram-kv-mib 9984
|
||||
--max-num-seqs 64 --max-num-batched-tokens 65536
|
||||
ARGS+=(--port 7770 --vram-kv-mib 3072
|
||||
--max-num-seqs 4 --max-num-batched-tokens 32768
|
||||
--prefix-cache-dir /data/calib-kvcache)
|
||||
;;
|
||||
probe)
|
||||
|
||||
Reference in new issue
Block a user