feat(frank-archon): four decode slots over a >=700k-token KV pool
Frank's order: 4 slots, a 700k pool, round UP if the resolver cannot land it exactly. The cache target is MAX_SEQS x MAX_MODEL_LEN, so the demand is 800,000 fp8 tokens; the resolver funds ~91-93 % of demand (three boots of evidence on the endpoint page), which clears 700k, and the VRAM has to come from the expert plane -- which has to go somewhere, so HOST_POOL_MIB moves with it. expert_vram_floor (core/place/planner.cpp:100-104) is exactly plane + 2 units - host_room, and the live boot ledger pins that plane at 67.60 GiB a rank (23.73 GiB floor at 44,928 of room), so 49,152 drops the floor to 19.61 GiB and leaves the cache ~6.9 GiB: enough for the whole target, with the leftover returned to the experts. Costs, both real: pinned host RAM 87.75 -> 96.0 GiB (the pool is pinned once per rank) and ~3.2 GiB a card of expert residency, read over the x8 Gen5 link when the router picks a host-resident expert. The boot ledger's vram_kv line is the referee; if it comes back under 700,000 tokens HOST_POOL_MIB goes up 1024 and this redeploys. TTM_MIN_MIB 150,000 -> 162,000 for the new pin ledger (2 x pool + PLE 48,824 + prefix 8,192 + MNBT 2,048 + the auto checkpoint store, now 40 slots of 112 MiB = 161,848 MiB), and the bats ledger guard now counts the checkpoint store because it grows with the slot count. The two env-value assertions in that seam are bumped with the values they guard.
This commit is contained in:
1 parent
aa75b45543
commit
1bb6399524
3 files changed
+40
-32
No files matched your search
@@ -15,9 +15,9 @@
|
||||
# log states every budget before anything is allocated (weights vs KV
|
||||
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
||||
# the P2P answer per pair) — read it after every resize.
|
||||
# Deployed shape (2026-10-19, radiance 1.8.1): TWO decode slots x 200,000 tokens,
|
||||
# fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in pinned host
|
||||
# RAM, PLE pinned in RAM. The per-card split between the expert slab and the KV
|
||||
# Deployed shape (2026-10-21, radiance 1.8.1): FOUR decode slots x 200,000 tokens over a
|
||||
# >= 700,000-token fp8 KV pool, 48 GiB/rank pinned expert pool, MTP depth 2, prefix cache
|
||||
# in pinned host RAM, PLE pinned in RAM. The per-card split between the expert slab and the KV
|
||||
# pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair and the budget
|
||||
# resolver derives it from headroom, the cache bound and the host pool, so the slab
|
||||
# size and the pool's token count are LEDGER lines. After every boot read them, plus
|
||||
@@ -27,9 +27,9 @@
|
||||
# pooled experts are read synchronously) — and the cache's session backing.
|
||||
#
|
||||
# The root unit (scripts/frank-archon.service, User=dank,
|
||||
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
|
||||
# the 47.7 GiB PLE table + 8 GiB prefix host tier — ~147 GiB of pinned
|
||||
# pages, which is ALSO why this script refuses to start when ttm's
|
||||
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 48 GiB expert pools + the 47.7 GiB PLE
|
||||
# table + 8 GiB prefix host tier + the checkpoint store (slots x 112 MiB) — ~159 GiB of
|
||||
# pinned pages, which is ALSO why this script refuses to start when ttm's
|
||||
# pages_limit/pages_max cap is under FRANK_ARCHON_TTM_MIN_MIB (see below).
|
||||
set -euo pipefail
|
||||
|
||||
|
||||
Reference in new issue
Block a user