perf(run-swift15-bf16): calib at the vendor big-step shape
MNBT 65536 / seqs 8: MOE-W4.md:244 measures 640 tok/s at a 32k step vs 930 at 55k — the KLD record's own rate confirmed step size drives disk-tier prefill. 131k resident tokens fits the 3072 pool (~237k tokens/rank at fp8). Cuts the 10M-token calibration from ~4.3 h to ~3 h.
This commit is contained in:
@@ -63,12 +63,16 @@ case "$MODE" in
|
||||
# unparameterisable threshold. Grams are bf16 activations — engine-invariant.
|
||||
# Same KV ceiling as kld: the disk-tier weights floor (9.57 GiB) plus
|
||||
# ~18 GiB of fixed per-rank claims cap the pool near 3.5 GiB on a 32 GiB
|
||||
# card. 4 slots x 16k fp8 fits; prefill rate is disk-bound, so fewer
|
||||
# resident slots only queues pushes, it does not slow the record.
|
||||
# card — pool minus the 2.17 GiB fixed holds ~237k fp8 tokens a rank, so
|
||||
# the vendor's big-step calibration shape is affordable: MOE-W4.md:244
|
||||
# measures 640 tok/s at a 32k step against 930 at 55k, and the doc's own
|
||||
# run drives MNBT 65536. 8 slots x 16k = 131k resident tokens fits the
|
||||
# pool; the step batches 65k. KLD's observed rate confirmed step size,
|
||||
# not slot count, is the throughput lever.
|
||||
IMAGE="stilldeadcode/radiance@$PIN_120"
|
||||
mkdir -p "$CALIB_DIR"
|
||||
ARGS+=(--port 7770 --vram-kv-mib 3072
|
||||
--max-num-seqs 4 --max-num-batched-tokens 32768
|
||||
--max-num-seqs 8 --max-num-batched-tokens 65536
|
||||
--prefix-cache-dir /data/calib-kvcache)
|
||||
;;
|
||||
probe)
|
||||
|
||||
Reference in New Issue
Block a user