frank-small: pool 540672 (3 x 180224) so the card keeps ~2.7 GiB free

Measured after the first 3-slot deploy: 3 x 196,608 booted (n_slots=3,
n_ctx_slot=196608, multimodal loaded) but used 22,972/24,576 MiB - 1,153 MiB
free, outside the 2-3 GiB band. The two measurements now recorded in
frank-small.env price the vision tower at ~1.4 GiB (non-KV 7,390 -> 8,859
MiB), so the pool drops 49,152 tokens (16,384/slot, 1,176 MiB) to land ~2.7
GiB free. The bats budget gate is recalibrated to the measured 9 GiB non-KV
cost, which now rejects the 3 x 196,608 shape instead of blessing it.
This commit is contained in:
Frank committed 2026-10-02 03:17:35 -07:00
1 parent a02850203d
commit 7fe881de01
4 files changed
+44 -37

No files matched your search

+16 -13
View File
@@ -98,7 +98,7 @@ argv_value() {
done
}
@test "the shipped env serves the fleet shape: three 196608-token contexts" {
@test "the shipped env serves the fleet shape: three 180224-token contexts" {
serve_argv
[ "$status" -eq 0 ]
[ "$(argv_value --alias)" = "frank-small" ]
@@ -112,7 +112,7 @@ argv_value() {
ctx="$(argv_value --ctx-size)"
par="$(argv_value --parallel)"
[ $((ctx % par)) -eq 0 ]
[ $((ctx / par)) -eq 196608 ]
[ $((ctx / par)) -eq 180224 ]
[ $((ctx / par)) -le 262144 ]
}
@@ -128,20 +128,23 @@ argv_value() {
@test "the KV budget leaves 2-3 GiB of the 3090 free" {
# Rate: K f16 (16 KiB/token) + V q8_0 (~8.5, budgeted 25 with slack).
# The non-KV half is MEASURED on this card on 2026-10-02, not guessed:
# weights 6.4 + projector 0.85 + compute buffers/CUDA graphs ~0.8 => 8.
# Two asserts, both about output the operator sees: the pool must still
# fit with 2 GiB spare, and it must be large enough that 5 GiB spare is
# NOT reachable — an undersized pool is as wrong as an oversized one.
# Arithmetic, not a boot log: change CTX_SIZE and this bites before the
# deploy does. The same pool with f16 V (34 KiB/token) lands past the
# card, which is why the contract is asymmetric.
# The non-KV half is MEASURED on this card on 2026-10-02: with the
# projector loaded the engine reported 22,972 MiB used against 14,113
# MiB of KV, so weights + vision tower + DeltaNet state + compute
# buffers + CUDA graphs = 8,859 MiB => 9 GiB. Two asserts, both about
# the output the operator sees: the pool must still fit with 2 GiB
# spare, and it must be large enough that 5 GiB spare is NOT reachable
# (an undersized pool is as wrong as an oversized one). Arithmetic, not
# a boot log: change CTX_SIZE and this bites before the deploy does —
# it rejects the 3 x 196,608 pool that measured only 1.1 GiB free. The
# same pool with f16 V (34 KiB/token) lands past the card, which is why
# the contract is asymmetric.
ctx="$(sed -nE 's/^CTX_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"
kv_gib=$((ctx * 25 / 1048576))
[ $((kv_gib + 8 + 2)) -le 24 ]
[ $((kv_gib + 8 + 5)) -gt 24 ]
[ $((kv_gib + 9 + 2)) -le 24 ]
[ $((kv_gib + 9 + 5)) -gt 24 ]
f16v_gib=$((ctx * 34 / 1048576))
[ $((f16v_gib + 8)) -gt 24 ]
[ $((f16v_gib + 9)) -gt 24 ]
}
@test "no speculative draft ships by default" {