llm(frank-archon): 3 x 200,000, pool cut to the contract, slab +1,792 MiB

Frank's call: three slots, and spend the KV the third slot doesn't need on
resident experts. `--max-num-seqs 3`, `--vram-kv-mib 6912 -> 5120`,
`--vram-weights-mib 22208 -> 24000` — the pair is still exactly the measured
29,120 MiB/card ceiling. The pool funds 608,027 tokens against the 600,000-token
contract (3 x 200,000, +1.3 %), and the slab gains 8.1 %.

The direction is the 2026-10-11 A/B: prefill is slab-limited, KV beyond the
contract buys nothing. At fp8 the slab curve measured on this host is
22,208 -> 3,397 t/s and 26,071 -> 4,379 t/s at 43.5k, i.e. ~0.25 t/s per MiB, so
24,000 predicts ~3,850 t/s single-stream. Falsifiable; the probe is the check.

The cost is named, not hidden: a FOURTH full-height conversation no longer fits
(4 x 200,000 needs 6,601 MiB of pool) and is refused as too large to ever fit —
radiance never truncates. Reverting to four slots is one env pair.

Seam: two new arithmetic gates, because both failures are boot-then-400 shape
faults that no flag check catches — the pair must respect the per-card ceiling,
and the pool must back seqs x max-model-len at the dtype's measured rate plus
the dtype-independent terms (110 MiB a sequence of recurrent state, 448 MiB of
prefix checkpoints).
This commit is contained in:
Frank
2026-10-13 20:00:39 -07:00
parent 7daa886dd6
commit 066813ea95
3 changed files with 82 additions and 37 deletions
+27 -3
View File
@@ -133,14 +133,14 @@ nosudo_bin() {
grep -q '^FRANK_ARCHON_MAX_MODEL_LEN=200000$' "$ENV_FILE"
ml=$(awk -F= '/^FRANK_ARCHON_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE")
[ "$ml" -ge 177356 ]
# Concurrency: 4 slots x the served window is the capacity contract, and
# the fused TP all-reduce is declared over seqs x (1 + spec) = 12 rows,
# Concurrency: 3 slots x the served window is the capacity contract, and
# the fused TP all-reduce is declared over seqs x (1 + spec) = 9 rows,
# under the 25-row bound at which the wht6 wire stays bit-exact. The
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
# tied inside the +-3% noise floor — slots buy concurrency, not speed.
grep -q '^FRANK_ARCHON_MAX_SEQS=4$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_MAX_SEQS=3$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_SPEC_TOKENS=2$' "$ENV_FILE"
@@ -154,6 +154,30 @@ nosudo_bin() {
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
grep -q '^FRANK_ARCHON_VRAM_KV_MIB=[1-9]' "$ENV_FILE"
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
# Two arithmetic contracts the engine ENFORCES at request/boot time, so
# they are gates and not lore (a bad pair boots, then 400s every long
# request — the 2026-10-11 pool of 6,912 MiB at seqs 4 accepted 200k-token
# chats and refused them as "too large to ever fit"):
# 1. vram_weights + vram_kv <= the measured 29,120 MiB/card ceiling
# (31,856 card - 1,710 consumed outside both budgets - 1,024 kept free)
# 2. the pool must back seqs x max-model-len at the dtype's rate, plus
# the dtype-INDEPENDENT terms that live in the pool: ~110 MiB of
# recurrent state a sequence and 448 MiB of prefix checkpoints.
# Rates are the two boot ledgers' group tables: 7,488 B/token fp8,
# 14,144 B/token bf16 (the QSA block key stays 3.25 KiB at both).
w=$(awk -F= '/^FRANK_ARCHON_VRAM_WEIGHTS_MIB=/{print $2}' "$ENV_FILE")
kv=$(awk -F= '/^FRANK_ARCHON_VRAM_KV_MIB=/{print $2}' "$ENV_FILE")
seqs=$(awk -F= '/^FRANK_ARCHON_MAX_SEQS=/{print $2}' "$ENV_FILE")
dt=$(awk -F= '/^FRANK_ARCHON_KV_DTYPE=/{print $2}' "$ENV_FILE")
[ $(( w + kv )) -le 29120 ]
case "$dt" in
fp8) rate=7488 ;;
bf16) rate=14144 ;;
*) return 1 ;;
esac
tokens=$(( seqs * ml ))
need=$(( tokens * rate / 1048576 + 110 * seqs + 448 ))
[ "$kv" -ge "$need" ]
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
grep -q '^FRANK_ARCHON_PREFIX_HOST_MIB=8192$' "$ENV_FILE"