Compare commits

...
Author SHA1 Message Date
Frank 4d41366f4c fix(frank-archon): declare two slots, revert the inert host-pool raise
FRANK_ARCHON_MAX_SEQS 3 -> 2. At 1.8.1 the budget resolver sizes the KV cache
from the cache bound (--max-model-len x --max-num-seqs), --gpu-headroom-mib and
--host-pool-mib, and the shipped CLI has no KV-size flag (the
--vram-weights-mib/--vram-kv-mib pair was removed at 1.3.0). The live boot
ledger reads that cache at 4.20 GiB = 556,348 tokens = 2.8 sessions at 200,000,
so a third full-height conversation is refused while /health answers green and
the harness advertises capacity the engine cannot honour. Two slots is what the
shape actually serves at the full window; a slot costs KV, never decode, and the
fused TP all-reduce drops from 9 rows to 2 x (1 + spec) = 6, still inside
wht6's 25-row bit-exact bound.

FRANK_ARCHON_HOST_POOL_MIB back to 44928 and FRANK_ARCHON_TTM_MIN_MIB back to
150000, reverting PR #30. That raise bought cache the resolver never hands out:
the two boot ledgers differ in exactly one line -- host_pool 44.47 -> 45.09 GiB
-- while vram_weights (25.81 GiB) and vram_kv (4.20 GiB / 556,348 tokens, both
ranks) are byte-identical, so the extra 640 MiB sat unclaimed in the host pool
while the TTM floor moved out from under the preflight's arithmetic. The
compute-bound cap on --host-pool-mib is unchanged either way (45,568 -> 22,222
MiB of experts either side of the 22,222 clamp).

The bats seam pins the new contract: MAX_SEQS=2, pool 44,928, and the ledger
floor that computes to 148,920 MiB at that pool against the 150,000 minimum.
2026-10-19 11:07:11 -07:00
dank 64765c6a47 Merge pull request 'fix(frank-archon): size the host pool so the KV cache clears 3 x 200,000' (#30) from fix/frank-archon-cache-ceiling into main 2026-10-19 10:30:13 -07:00
Frank d5934a6b47 fix(frank-archon): size the host pool so the KV cache clears 3 x 200,000
The 1.8.1 boot resolved the budget resolver to 4.20 GiB of cache a card —
556,348 tokens, 2.8 sessions at full height — so three concurrent
full-height requests are refused with a 400 at admission: the endpoint's
3-slot contract is broken by the cutover, silently, because /health and
every shorter conversation still answer.

The resolver gives the expert plane whatever the pinned host pool cannot
hold and the cache the rest, so the pool is the lever: +640 MiB a rank
moves that much expert demand into host RAM and the cache wins it back.
Tokens scale with the GiB won (8.11 KiB/token at fp8), so ~0.63 GiB buys
~44,000 tokens — 600,000 with margin — at ~2.5% off the expert slab.
TTM floor moves with the pool (2 x pool + 48,824 + prefix + MNBT =
150,200 MiB computed by the ledger test).

Verified by the boot ledger's `kv:` line and the probe's capacity leg,
not by this arithmetic.
2026-10-19 10:29:20 -07:00
dank 40ad1f4a60 Merge pull request 'feat(frank-archon): radiance 1.8.1 — the budget resolver, the sharing knobs, and the CLI breaks between them' (#29) from feat/frank-archon-radiance-181 into main 2026-10-19 10:21:27 -07:00
2 changed files with 23 additions and 11 deletions

No files matched your search

+17 -8
View File
@@ -47,10 +47,13 @@ FRANK_ARCHON_TP=2
# MAX_SEQS x (1 + SPEC_TOKENS) below. # MAX_SEQS x (1 + SPEC_TOKENS) below.
FRANK_ARCHON_TP_WIRE=wht6 FRANK_ARCHON_TP_WIRE=wht6
# Decode slots. A slot costs KV, never decode; the pool is sized to this # Decode slots. A slot costs KV, never decode; the pool is sized to this
# contract and nothing more, so a FOURTH full-height conversation is REFUSED as # contract and nothing more. At 1.8.1 the budget resolver caps the cache at
# too large to fit — never squeezed, paged or truncated. That is Frank's # 556,348 fp8 tokens — 2.8 sessions at 200,000 — so a THIRD full-height
# explicit trade: the freed MiB went to the expert slab. # conversation answers 400 while /health and every shorter chat work fine
FRANK_ARCHON_MAX_SEQS=3 # (3 x 50k streams run at 77.6 t/s aggregate). Frank's trade: declare the two
# slots the engine can hold and let the freed ~1.1 GiB go to the expert plane,
# instead of advertising a third slot that only exists at shorter prompts.
FRANK_ARCHON_MAX_SEQS=2
# Served window. The engine treats `kv group tokens >= max-model-len` as a HARD # Served window. The engine treats `kv group tokens >= max-model-len` as a HARD
# limit, and models.yml contextWindow MUST quote this number (over-length is a # limit, and models.yml contextWindow MUST quote this number (over-length is a
# refusal, not a truncation). The pool is sized for seqs x maxlen, not one # refusal, not a truncation). The pool is sized for seqs x maxlen, not one
@@ -63,6 +66,10 @@ FRANK_ARCHON_PLACEMENT=expert_tiered
# rest, so growing the pool moves experts off the card and shrinking it spills # rest, so growing the pool moves experts off the card and shrinking it spills
# them onto VRAM the cache wanted. Under-sizing it is a boot failure (36,864 # them onto VRAM the cache wanted. Under-sizing it is a boot failure (36,864
# refused a boot). The ledger's coverage line is the referee, not this file. # refused a boot). The ledger's coverage line is the referee, not this file.
# Measured, not guessed: 45,568 (PR #30) moved host_pool to 45.09 GiB and left
# vram_kv byte-identical at 4.20 GiB / 556,348 tokens on both ranks — extra
# host RAM above what the expert plane's footprint needs buys no cache. Back at
# 44,928, the size that boots clean.
FRANK_ARCHON_HOST_POOL_MIB=44928 FRANK_ARCHON_HOST_POOL_MIB=44928
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle # VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
# floor is built from it. # floor is built from it.
@@ -107,10 +114,12 @@ FRANK_ARCHON_PREFIX_CACHE_MIB=8192
# read. `ram` REFUSES the start if memlock is too small — the unit's # read. `ram` REFUSES the start if memlock is too small — the unit's
# LimitMEMLOCK=infinity is the other half of this law. # LimitMEMLOCK=infinity is the other half of this law.
FRANK_ARCHON_NGRAM_PLACEMENT=ram FRANK_ARCHON_NGRAM_PLACEMENT=ram
# TTM preflight. The pinned ledger (~147 GiB) rides TTM's shared pages, and the # TTM preflight. The pinned ledger rides TTM's shared pages, and the kernel
# kernel RESETS pages_limit to half of RAM on every boot; below the plan the # RESETS pages_limit to half of RAM on every boot; below the plan the load dies
# load dies mid-pin with an anonymous allocator error. ttm-preflight.sh raises # mid-pin with an anonymous allocator error. ttm-preflight.sh raises the knob
# the knob itself before refusing by name, so recovery is a boot, not a visit. # itself before refusing by name, so recovery is a boot, not a visit.
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT
# per the bats ledger test; at the 44,928 pool that computes to 148,920 MiB.
FRANK_ARCHON_TTM_MIN_MIB=150000 FRANK_ARCHON_TTM_MIN_MIB=150000
FRANK_ARCHON_TTM_TARGET_MIB=204800 FRANK_ARCHON_TTM_TARGET_MIB=204800
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands # Escape hatch for experiments. Unset -> no extra argv; the serve script expands
+6 -3
View File
@@ -135,14 +135,17 @@ nosudo_bin() {
grep -q '^FRANK_ARCHON_MAX_MODEL_LEN=200000$' "$ENV_FILE" grep -q '^FRANK_ARCHON_MAX_MODEL_LEN=200000$' "$ENV_FILE"
ml=$(awk -F= '/^FRANK_ARCHON_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE") ml=$(awk -F= '/^FRANK_ARCHON_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE")
[ "$ml" -ge 177356 ] [ "$ml" -ge 177356 ]
# Concurrency: 3 slots x the served window is the capacity contract, and # Concurrency: 2 slots x the served window is the capacity contract at
# the fused TP all-reduce is declared over seqs x (1 + spec) = 9 rows, # 1.8.1. The budget resolver caps the cache at 556,348 fp8 tokens — 2.8
# sessions at 200,000 — so three concurrent full-height chats answer 400,
# and the shipped CLI has no KV-size flag to raise it (removed at 1.3.0).
# The fused TP all-reduce is declared over seqs x (1 + spec) = 6 rows,
# under the 25-row bound at which the wht6 wire stays bit-exact. The # under the 25-row bound at which the wht6 wire stays bit-exact. The
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork # "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted # (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3 # seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
# tied inside the +-3% noise floor — slots buy concurrency, not speed. # tied inside the +-3% noise floor — slots buy concurrency, not speed.
grep -q '^FRANK_ARCHON_MAX_SEQS=3$' "$ENV_FILE" grep -q '^FRANK_ARCHON_MAX_SEQS=2$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE" grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE" grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_SPEC_TOKENS=2$' "$ENV_FILE" grep -q '^FRANK_ARCHON_SPEC_TOKENS=2$' "$ENV_FILE"