dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-secondary: ubatch 2048 (the second rung inside the 112k headroom)

Browse Source 
Frank's order the same night. The 1024->2048 rung measured +4.6 %
prefill for 616 MiB on the 3090 (2026-10-05 boot pair); the 112k cut
left 1,751 MiB free, so it rides with ~1.1 GiB predicted margin.
Decode is ubatch-invariant; batch 4096 >= 2048 keeps the seam happy.
This commit is contained in:
Frank
2026-10-06 18:30:35 -07:00
parent 44e7a8dfaf
commit d8da7a9eb7
2 changed files with 9 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/frank_secondary_unit.bats
+3 -3
View File
@@ -146,10 +146,10 @@ argv_value() {
# RCO-IQ3_XXS weights 10,148 MiB (10,442,827,392 B) — 1.56 GiB LIGHTER
# than the IQ3_S-mtp predecessor — + vision tower ~1.4 GiB (931,145,888
# B BF16 mmproj at the measured ~1.45x cost law) + compute buffers at
# ubatch 1024 + CUDA graphs + TWO DeltaNet states (~150 MiB each) + the
# ubatch 2048 + CUDA graphs + TWO DeltaNet states (~150 MiB each) + the
# MTP draft ctx => 14 GiB budgeted. Two 114,688 slots are the
# 2026-10-06 evening shape (the lighter per-slot funds the ubatch
# increase); a THIRD slot must not fit. Arithmetic, not a boot log:
# 2026-10-06 late shape (the lighter per-slot funds the ubatch
# rungs); a THIRD slot must not fit.
# change CTX_SIZE and this bites before the deploy does; the boot's
# nvidia-smi free is the measured truth.
ctx="$(sed -nE 's/^CTX_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"