docs(scripts): correct frank-archon's step-sharing and slot-count comments
The env comment claimed BUSY_PREFILL_CHUNK=auto protects decode rows; the 1.8.1 source (core/sched/scheduler.cpp busy_cap) reserves 1+spec rows per decoding sequence and 1/8 of the budget per waiting request, and refuses to shrink the chunk for decoders alone -- auto is a prefill-speed dial. The interleave comment carried a 512 that is not this build's auto value (0 until the held-step counter asks; 64 in help mode). The serve header still described three decode slots; MAX_SEQS has been 2 since the 2026-10-19 reslice. No behavior change.
This commit is contained in:
1 parent
961713e92d
commit
687dbfbabe
2 files changed
+25
-15
No files matched your search
@@ -15,13 +15,16 @@
|
||||
# log states every budget before anything is allocated (weights vs KV
|
||||
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
||||
# the P2P answer per pair) — read it after every resize.
|
||||
# Deployed shape (2026-10-19, radiance 1.8.1): THREE decode slots x 200,000
|
||||
# tokens, fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in
|
||||
# pinned host RAM, PLE pinned in RAM. The per-card split between the expert slab
|
||||
# and the KV pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair
|
||||
# and the budget resolver derives it from headroom, the cache bound and the host
|
||||
# pool, so the slab size and the pool's token count are LEDGER lines: after every
|
||||
# boot read them out of the log and check the pool still backs 3 x 200,000.
|
||||
# Deployed shape (2026-10-19, radiance 1.8.1): TWO decode slots x 200,000 tokens,
|
||||
# fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in pinned host
|
||||
# RAM, PLE pinned in RAM. The per-card split between the expert slab and the KV
|
||||
# pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair and the budget
|
||||
# resolver derives it from headroom, the cache bound and the host pool, so the slab
|
||||
# size and the pool's token count are LEDGER lines. After every boot read them, plus
|
||||
# the two lines that size a prefill step — "activation arena: ... lends ... while
|
||||
# steps are at most N tokens" (the arena levels the expert slab borrows) and
|
||||
# "prefill staging: ... a step of more than N tokens ..." (the chunk size below which
|
||||
# pooled experts are read synchronously) — and the cache's session backing.
|
||||
#
|
||||
# The root unit (scripts/frank-archon.service, User=dank,
|
||||
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
|
||||
|
||||
Reference in new issue
Block a user