docs(frank-archon): the slab is a slope, and the ceiling was mine not the engine's
Measured A/B on the shipped pair 25600/5120: A-cold 43.5k prefill 3,290 -> 3,584 t/s (+8.9 %) and the 3-way 197k aggregate 3,051 -> 3,589 t/s (+17.6 %, wall 193.7 -> 164.5 s). The prediction I shipped with the arm -- slab is a threshold below ~26 GiB, so 1,600 MiB buys ~0 -- is falsified by the measurement it asked for. Two slab points inside noise of EACH OTHER (22,208 / 24,000) are not a plateau. The cliff is above 25,600: 26,071 measured 4,379 t/s. Ceiling re-derived from the engine's own budget table (card 32,624 MiB, pair at 29,120 settling 2,058 MiB free against its 209.19 MiB floor => 30,969 claimable). The 29,120 bound was my 1 GiB \`keep free\` reserve; the seam now derives the number, so the next resize starts from the engine's arithmetic. Ledger for the record: slab 16,235 -> 17,548 slots while the pinned host pool hands the same experts back (8,893 -> 7,580) -- the transfer is the mechanism, the pinned pool is the slow path. Capacity contract live-verified at 3 slots: 3 x 199,984-token prompts accepted (599,952 tokens), fourth refused as too large to ever fit. Probe law learned: the oversized-prompt refusal names the prompt's token count BEFORE prefill (free calibration) and fires on prompt tokens ALONE, not on prompt + max_tokens; the window is ~514,286 chars at 2.5714 chars/token.
This commit is contained in:
1 parent
09a0251d2d
commit
2581b8d4b6
5 files changed
+153
-42
No files matched your search
@@ -32,6 +32,41 @@
|
||||
aggregate prefill, no preemption; prefix cache verified (repeat answered in
|
||||
1.0 s, `cache_n=43008`). Playwright e2e 32 passed + 1 flaky (the known Plex
|
||||
homepage retry); archon bats seam 21/21 including three new ttm-gate tests.
|
||||
- **Re-cut twice the same evening (Frank's calls), each read off the ledger.**
|
||||
4 → 3 slots, pool 6,912 → 5,120: three slots need 5,063 MiB and the 6,912 pool
|
||||
was funding 858,969 tokens — 43 % over the contract. Then slab
|
||||
24,000 → 25,600, claiming 1,600 of the 1,849 MiB the engine leaves above its
|
||||
OWN settle floor (`card 0 settled at 2.01 GiB free against 209.19 MiB (the
|
||||
96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)`): the
|
||||
real ceiling is 30,969 MiB, not the 29,120 my 1 GiB "keep free" reserve
|
||||
invented on 2026-10-11. The pair now spends 30,720 and settles
|
||||
`unclaimed 477.30 MiB` (cards ~336 MiB free).
|
||||
- **The prediction shipped with that arm was falsified by its own measurement,
|
||||
which is the whole point of banking the A/B.** Claim: the slab is a THRESHOLD
|
||||
below ~26 GiB (22,208 → 3,397 and 24,000 → 3,290 t/s are inside noise of each
|
||||
other), so 1.6 GiB buys ~0. Measured at 25,600: **3,584 t/s** single-stream
|
||||
cold at 43.5k (+8.9 %) and **3,589 t/s** 3-way aggregate at 197k (wall
|
||||
193.7 → 164.5 s, +17.6 %) — slab 16,235 → 17,548 slots while the pinned host
|
||||
pool hands the same experts back (8,893 → 7,580). The slab is a SLOPE; two
|
||||
points inside noise of each other are not a plateau. The cliff is above
|
||||
25,600: 26,071 measured 4,379 t/s (+22 % for +471 MiB), unreachable at three
|
||||
slots (26,112 + 5,120 > the ceiling) and free at two.
|
||||
- **Capacity verified live at 3 slots:** three concurrent 199,984-token prompts
|
||||
accepted and completed (599,952 tokens resident), ledger
|
||||
`backs 3 sequences all at 200000 tokens at once` at 607,740 tokens against the
|
||||
law's 608,027. A fourth full-height request is refused as too large to ever
|
||||
fit — radiance never truncates. Probe law: the oversized-prompt counter is
|
||||
FREE (the refusal fires when PROMPT tokens alone exceed the window —
|
||||
`prompt: 202,777 prompt tokens does not fit the 200000 token context`) and the
|
||||
window is ~514,286 characters at the measured 2.5714 chars/token, so a
|
||||
calibration probe below that pays a real prefill instead of reporting.
|
||||
- **Open call for Frank:** two slots (pool 3,072 + slab 27,897 = the ceiling
|
||||
exactly) crosses the cliff for the +22 %, and costs the third concurrent 200k
|
||||
conversation. Untested arm; capacity decision, not tuning.
|
||||
- Seam: the pair ceiling is derived from the engine's budget table (30,969) and
|
||||
owns that arithmetic alone; the KV-pool contract (seqs × window at the dtype's
|
||||
measured rate + 110 MiB a sequence + 448 MiB of prefix checkpoints) keeps its
|
||||
own test. 371/371 tests and e2e 33/33 on both deploys.
|
||||
- **Power-failure casualties, NOT fixed (outside the archon window):**
|
||||
`mnt-md0.mount` failed at boot, so `/mnt/md0` is an empty mountpoint (the
|
||||
raid6 array itself is healthy, 7/7; fstab has `nofail` so boot continued) —
|
||||
|
||||
Reference in new issue
Block a user