dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(run-swift15-bf16): per-mode engine pins + explicit KV pools

Browse Source 
kld stays on 1.0.13 (record on the scoring engine) with --vram-kv-mib
4608 + 16 slots: the ratio-derived pool came up SHORT by 217 MiB at
32x16k. calib moves to 1.2.0: the RADIANCE_CALIB_TOKENS tap is a 1.2.0
qwen4exp plugin feature (1.0.13 plugin has the DIR half only), and
Grams are bf16 activations - engine-invariant. --vram-kv-mib 9984
covers 64x16k fp8 slots.
This commit is contained in:
Frank
2026-10-06 22:31:17 -07:00
parent 9bc40fd899
commit 4048dc8630
1 changed files with 16 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/run-swift15-bf16.sh
+16 -3
View File
@@ -21,7 +21,8 @@
set -uo pipefail set -uo pipefail
MODE="${1:?usage: run-swift15-bf16.sh kld|calib}" MODE="${1:?usage: run-swift15-bf16.sh kld|calib}"
IMAGE="stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e" PIN_1013=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
PIN_120=sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa
RAD=/home/dank/models/rad/qwen4exp-swift15-bf16.rad RAD=/home/dank/models/rad/qwen4exp-swift15-bf16.rad
C=swift15ref C=swift15ref
LOG=/home/dank/logs/swift15_bf16.log LOG=/home/dank/logs/swift15_bf16.log
@@ -44,16 +45,28 @@ ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad
case "$MODE" in case "$MODE" in
kld) kld)
# 1.0.13: the engine that will serve the candidate — the reference is
# recorded on the same engine that scores it. The ratio-derived KV pool
# (1.95 GiB) is SHORT by 217 MiB at 32x16k; the pair is explicit: 16 slots
# need 2.17 fixed + 1.87 GiB of fp8 KV, 4608 MiB carries the margin.
IMAGE="stilldeadcode/radiance@$PIN_1013"
mkdir -p "$KLD_REF_DIR" mkdir -p "$KLD_REF_DIR"
ARGS+=(--kld-record /data/kld-ref --kld-corpus /data/kld.jsonl ARGS+=(--kld-record /data/kld-ref --kld-corpus /data/kld.jsonl
--max-num-seqs 32 --max-num-batched-tokens 32768) --vram-kv-mib 4608
--max-num-seqs 16 --max-num-batched-tokens 32768)
;; ;;
calib) calib)
# 1.2.0: the calibration tap (RADIANCE_CALIB_DIR + RADIANCE_CALIB_TOKENS)
# lives in the 1.2.0 qwen4exp plugin; 1.0.13 has only the DIR half with an
# unparameterisable threshold. Grams are bf16 activations — engine-invariant.
IMAGE="stilldeadcode/radiance@$PIN_120"
mkdir -p "$CALIB_DIR" mkdir -p "$CALIB_DIR"
ARGS+=(--port 7770 --max-num-seqs 64 --max-num-batched-tokens 65536 ARGS+=(--port 7770 --vram-kv-mib 9984
--max-num-seqs 64 --max-num-batched-tokens 65536
--prefix-cache-dir /data/calib-kvcache) --prefix-cache-dir /data/calib-kvcache)
;; ;;
probe) probe)
IMAGE="stilldeadcode/radiance@$PIN_1013"
ARGS+=(--port 7770 --max-num-seqs 8 --max-num-batched-tokens 4096) ARGS+=(--port 7770 --max-num-seqs 8 --max-num-batched-tokens 4096)
;; ;;
*) echo "unknown mode: $MODE" >&2; exit 2 ;; *) echo "unknown mode: $MODE" >&2; exit 2 ;;