docs(scripts): correct frank-archon's step-sharing and slot-count comments

The env comment claimed BUSY_PREFILL_CHUNK=auto protects decode rows; the 1.8.1
source (core/sched/scheduler.cpp busy_cap) reserves 1+spec rows per decoding
sequence and 1/8 of the budget per waiting request, and refuses to shrink the
chunk for decoders alone -- auto is a prefill-speed dial. The interleave comment
carried a 512 that is not this build's auto value (0 until the held-step
counter asks; 64 in help mode). The serve header still described three decode
slots; MAX_SEQS has been 2 since the 2026-10-19 reslice. No behavior change.
This commit is contained in:
Frank
2026-10-20 22:07:55 -07:00
parent 961713e92d
commit 687dbfbabe
2 changed files with 25 additions and 15 deletions
+15 -8
View File
@@ -93,14 +93,21 @@ FRANK_ARCHON_SPEC_TOKENS=2
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
# Step sharing. A prefill chunk that shares a step with decoding sequences holds
# the device for the chunk's whole time, and every decode row in that step waits
# that long for its next token (the census: [[LLM Performance Diagnosis]] §9).
# `auto` leaves each decode row behind the chunk a row plus its draft window and
# keeps a share for whatever is waiting to be admitted; a number pins the chunk's
# cap; `off` is the pre-1.4 behaviour this endpoint ran for two eras, where the
# chunk took the step and admission never ran. Interleave inserts N decode-only
# steps after a chunk step taken beside decoders: it protects their cadence and
# holds a waiting request out of those steps, so it stays 0 until the post-migration
# decode-vs-prefill buckets say it is wanted.
# that long for its next token: measured at this shape, decode-only steps 15.5 ms
# against chunk steps 166 ms, which is the harness's ~146 t/s falling to ~11 t/s
# while another session prefills (law and census: [[frank-archon]]).
# `auto` is a PREFILL-SPEED dial, not a decoder dial: it reserves one row plus the
# draft window per decoding sequence (3 tokens) and an eighth of the budget per
# waiting request or waiting prefill, and the source states it will not halve for
# decode rows alone. Live proof: prefill_chunks_capped 5,018 across 62,386 chunk
# steps. A number is the only value that applies whenever the chunk shares a step,
# and it must not go under the prefill stager's smallest pass -- 1025 rows here,
# named on the boot line "prefill staging: ... a step of more than 1024 tokens" --
# because below it every routed layer reads its pooled experts synchronously, so
# 512 is a trap and the vendor measures the whole job slower there. Interleave
# inserts N decode-only steps after a chunk step taken beside decoders: it protects
# their cadence and holds a waiting request out of those steps. That is the decoder
# dial, and it is 0 until the decode-vs-prefill buckets ask for it.
FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto
FRANK_ARCHON_PREFILL_INTERLEAVE=0
# Prefix cache: pinned host memory only. The disk tier was removed outright
+10 -7
View File
@@ -15,13 +15,16 @@
# log states every budget before anything is allocated (weights vs KV
# split, the expert host pool's coverage of the worst case, the PLE pin,
# the P2P answer per pair) — read it after every resize.
# Deployed shape (2026-10-19, radiance 1.8.1): THREE decode slots x 200,000
# tokens, fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in
# pinned host RAM, PLE pinned in RAM. The per-card split between the expert slab
# and the KV pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair
# and the budget resolver derives it from headroom, the cache bound and the host
# pool, so the slab size and the pool's token count are LEDGER lines: after every
# boot read them out of the log and check the pool still backs 3 x 200,000.
# Deployed shape (2026-10-19, radiance 1.8.1): TWO decode slots x 200,000 tokens,
# fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in pinned host
# RAM, PLE pinned in RAM. The per-card split between the expert slab and the KV
# pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair and the budget
# resolver derives it from headroom, the cache bound and the host pool, so the slab
# size and the pool's token count are LEDGER lines. After every boot read them, plus
# the two lines that size a prefill step — "activation arena: ... lends ... while
# steps are at most N tokens" (the arena levels the expert slab borrows) and
# "prefill staging: ... a step of more than N tokens ..." (the chunk size below which
# pooled experts are read synchronously) — and the cache's session backing.
#
# The root unit (scripts/frank-archon.service, User=dank,
# LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +