perf(run-swift15-bf16): calib at the vendor big-step shape

MNBT 65536 / seqs 8: MOE-W4.md:244 measures 640 tok/s at a 32k step
vs 930 at 55k — the KLD record's own rate confirmed step size drives
disk-tier prefill. 131k resident tokens fits the 3072 pool (~237k
tokens/rank at fp8). Cuts the 10M-token calibration from ~4.3 h to
~3 h.
This commit is contained in:
Frank
2026-10-06 23:39:10 -07:00
parent fa858f4c10
commit 45de2fd47f
+7 -3
View File
@@ -63,12 +63,16 @@ case "$MODE" in
# unparameterisable threshold. Grams are bf16 activations — engine-invariant.
# Same KV ceiling as kld: the disk-tier weights floor (9.57 GiB) plus
# ~18 GiB of fixed per-rank claims cap the pool near 3.5 GiB on a 32 GiB
# card. 4 slots x 16k fp8 fits; prefill rate is disk-bound, so fewer
# resident slots only queues pushes, it does not slow the record.
# card — pool minus the 2.17 GiB fixed holds ~237k fp8 tokens a rank, so
# the vendor's big-step calibration shape is affordable: MOE-W4.md:244
# measures 640 tok/s at a 32k step against 930 at 55k, and the doc's own
# run drives MNBT 65536. 8 slots x 16k = 131k resident tokens fits the
# pool; the step batches 65k. KLD's observed rate confirmed step size,
# not slot count, is the throughput lever.
IMAGE="stilldeadcode/radiance@$PIN_120"
mkdir -p "$CALIB_DIR"
ARGS+=(--port 7770 --vram-kv-mib 3072
--max-num-seqs 4 --max-num-batched-tokens 32768
--max-num-seqs 8 --max-num-batched-tokens 65536
--prefix-cache-dir /data/calib-kvcache)
;;
probe)