frank-small: pool 540672 (3 x 180224) so the card keeps ~2.7 GiB free
Measured after the first 3-slot deploy: 3 x 196,608 booted (n_slots=3, n_ctx_slot=196608, multimodal loaded) but used 22,972/24,576 MiB - 1,153 MiB free, outside the 2-3 GiB band. The two measurements now recorded in frank-small.env price the vision tower at ~1.4 GiB (non-KV 7,390 -> 8,859 MiB), so the pool drops 49,152 tokens (16,384/slot, 1,176 MiB) to land ~2.7 GiB free. The bats budget gate is recalibrated to the measured 9 GiB non-KV cost, which now rejects the 3 x 196,608 shape instead of blessing it.
This commit is contained in:
1 parent
a02850203d
commit
7fe881de01
4 files changed
+44
-37
No files matched your search
+16
-13
@@ -98,7 +98,7 @@ argv_value() {
|
||||
done
|
||||
}
|
||||
|
||||
@test "the shipped env serves the fleet shape: three 196608-token contexts" {
|
||||
@test "the shipped env serves the fleet shape: three 180224-token contexts" {
|
||||
serve_argv
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(argv_value --alias)" = "frank-small" ]
|
||||
@@ -112,7 +112,7 @@ argv_value() {
|
||||
ctx="$(argv_value --ctx-size)"
|
||||
par="$(argv_value --parallel)"
|
||||
[ $((ctx % par)) -eq 0 ]
|
||||
[ $((ctx / par)) -eq 196608 ]
|
||||
[ $((ctx / par)) -eq 180224 ]
|
||||
[ $((ctx / par)) -le 262144 ]
|
||||
}
|
||||
|
||||
@@ -128,20 +128,23 @@ argv_value() {
|
||||
|
||||
@test "the KV budget leaves 2-3 GiB of the 3090 free" {
|
||||
# Rate: K f16 (16 KiB/token) + V q8_0 (~8.5, budgeted 25 with slack).
|
||||
# The non-KV half is MEASURED on this card on 2026-10-02, not guessed:
|
||||
# weights 6.4 + projector 0.85 + compute buffers/CUDA graphs ~0.8 => 8.
|
||||
# Two asserts, both about output the operator sees: the pool must still
|
||||
# fit with 2 GiB spare, and it must be large enough that 5 GiB spare is
|
||||
# NOT reachable — an undersized pool is as wrong as an oversized one.
|
||||
# Arithmetic, not a boot log: change CTX_SIZE and this bites before the
|
||||
# deploy does. The same pool with f16 V (34 KiB/token) lands past the
|
||||
# card, which is why the contract is asymmetric.
|
||||
# The non-KV half is MEASURED on this card on 2026-10-02: with the
|
||||
# projector loaded the engine reported 22,972 MiB used against 14,113
|
||||
# MiB of KV, so weights + vision tower + DeltaNet state + compute
|
||||
# buffers + CUDA graphs = 8,859 MiB => 9 GiB. Two asserts, both about
|
||||
# the output the operator sees: the pool must still fit with 2 GiB
|
||||
# spare, and it must be large enough that 5 GiB spare is NOT reachable
|
||||
# (an undersized pool is as wrong as an oversized one). Arithmetic, not
|
||||
# a boot log: change CTX_SIZE and this bites before the deploy does —
|
||||
# it rejects the 3 x 196,608 pool that measured only 1.1 GiB free. The
|
||||
# same pool with f16 V (34 KiB/token) lands past the card, which is why
|
||||
# the contract is asymmetric.
|
||||
ctx="$(sed -nE 's/^CTX_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"
|
||||
kv_gib=$((ctx * 25 / 1048576))
|
||||
[ $((kv_gib + 8 + 2)) -le 24 ]
|
||||
[ $((kv_gib + 8 + 5)) -gt 24 ]
|
||||
[ $((kv_gib + 9 + 2)) -le 24 ]
|
||||
[ $((kv_gib + 9 + 5)) -gt 24 ]
|
||||
f16v_gib=$((ctx * 34 / 1048576))
|
||||
[ $((f16v_gib + 8)) -gt 24 ]
|
||||
[ $((f16v_gib + 9)) -gt 24 ]
|
||||
}
|
||||
|
||||
@test "no speculative draft ships by default" {
|
||||
|
||||
Reference in new issue
Block a user