llm(frank-archon): claim the card's settling slack, slab 24000 -> 25600

The 2 GiB/card that \`rocm-smi\` shows free is not idle: the engine's budget
table calls it driver context + the 896 MiB activation arena + 96 MiB headroom,
and says the rest is `throughput on the floor`. Its own settle line is the
number that matters — `card 0 settled at 2.01 GiB free against 209.19 MiB (the
96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)` — so
1,849 MiB above the ENGINE's floor is claimable and the real ceiling is 30,969
MiB, not the 29,120 my 1 GiB reserve invented on 2026-10-11.

Shipped pair 25600/5120 = 30,720: +1,600 MiB of slab, ~458 MiB above the floor.
The slab is already saturated at 24,000 (mover took all of it, 451 KiB slack),
so this is the only direction left — KV is at its 3-slot floor (5,063 needed of
5,120).

Named prediction, because today's two points say the slab is a THRESHOLD and not
a slope (22,208 -> 3,397 t/s, 24,000 -> 3,290 t/s at 43.5k single-stream): this
arm buys ~1,600 resident expert slots and ~0 prefill. The point that moves
prefill is 26,071 (2026-10-11: 4,379 t/s), fundable only at two slots. Prefill
moving > +-3 % here falsifies the threshold reading.

Seam: the ceiling gate now derives from the engine's budget table (30,969) and
is the single owner of the pair arithmetic; the duplicate I put in the shape
test is removed.
This commit is contained in:
Frank
2026-10-13 20:18:41 -07:00
parent 066813ea95
commit 09a0251d2d
3 changed files with 46 additions and 51 deletions
+12 -31
View File
@@ -154,30 +154,6 @@ nosudo_bin() {
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
grep -q '^FRANK_ARCHON_VRAM_KV_MIB=[1-9]' "$ENV_FILE"
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
# Two arithmetic contracts the engine ENFORCES at request/boot time, so
# they are gates and not lore (a bad pair boots, then 400s every long
# request — the 2026-10-11 pool of 6,912 MiB at seqs 4 accepted 200k-token
# chats and refused them as "too large to ever fit"):
# 1. vram_weights + vram_kv <= the measured 29,120 MiB/card ceiling
# (31,856 card - 1,710 consumed outside both budgets - 1,024 kept free)
# 2. the pool must back seqs x max-model-len at the dtype's rate, plus
# the dtype-INDEPENDENT terms that live in the pool: ~110 MiB of
# recurrent state a sequence and 448 MiB of prefix checkpoints.
# Rates are the two boot ledgers' group tables: 7,488 B/token fp8,
# 14,144 B/token bf16 (the QSA block key stays 3.25 KiB at both).
w=$(awk -F= '/^FRANK_ARCHON_VRAM_WEIGHTS_MIB=/{print $2}' "$ENV_FILE")
kv=$(awk -F= '/^FRANK_ARCHON_VRAM_KV_MIB=/{print $2}' "$ENV_FILE")
seqs=$(awk -F= '/^FRANK_ARCHON_MAX_SEQS=/{print $2}' "$ENV_FILE")
dt=$(awk -F= '/^FRANK_ARCHON_KV_DTYPE=/{print $2}' "$ENV_FILE")
[ $(( w + kv )) -le 29120 ]
case "$dt" in
fp8) rate=7488 ;;
bf16) rate=14144 ;;
*) return 1 ;;
esac
tokens=$(( seqs * ml ))
need=$(( tokens * rate / 1048576 + 110 * seqs + 448 ))
[ "$kv" -ge "$need" ]
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
grep -q '^FRANK_ARCHON_PREFIX_HOST_MIB=8192$' "$ENV_FILE"
@@ -228,15 +204,20 @@ nosudo_bin() {
}
@test "the per-card VRAM pair stays inside the measured card ceiling" {
# Measured on Dankserver 2026-10-11: card 31,856 MiB, a boot consumes
# 1,710 MiB OUTSIDE both budgets (driver context, the 990 MiB activation
# arena, hardware queues, headroom, step staging) and the fp8 pair settled
# at 1,379 MiB free. Claiming more than 29,120 MiB in total is an
# allocation failure at graph capture, not a warning — so the KV pool and
# the weights slab are one budget, and this seam is the guard.
# The engine's OWN budget table, not a hand-picked reserve. Boot of
# 2026-10-13 at the pair 24000/5120: card 32,624 MiB (31.86 GiB), budgets
# claimed 29,120, `3.42 GiB is neither budgeted nor accounted — the
# driver's context, the activation arena and --gpu-headroom-mib come out of
# it; anything past those is throughput on the floor`, and `card 0 settled
# at 2.01 GiB free against 209.19 MiB (the 96.00 MiB --gpu-headroom-mib and
# 113.19 MiB kept for the static region)`. Claimable reserve above the
# engine's floor: 2,058 - 209 = 1,849 MiB, so the ceiling is 30,969 MiB.
# The 2026-10-11 bound of 29,120 ("keep >= 1 GiB free", card read as 31,856
# MiB) was MY reserve, which the engine never asked for; the shipped pair
# spends the difference and lands ~458 MiB above the floor.
w=$(awk -F= '/^FRANK_ARCHON_VRAM_WEIGHTS_MIB=/{print $2}' "$ENV_FILE")
kv=$(awk -F= '/^FRANK_ARCHON_VRAM_KV_MIB=/{print $2}' "$ENV_FILE")
[ $(( w + kv )) -le 29120 ]
[ $(( w + kv )) -le 30969 ]
}
@test "the serve script wires the engine flags from the env" {