docs(frank-archon): the slab is a slope, and the ceiling was mine not the engine's

Measured A/B on the shipped pair 25600/5120: A-cold 43.5k prefill 3,290 ->
3,584 t/s (+8.9 %) and the 3-way 197k aggregate 3,051 -> 3,589 t/s (+17.6 %,
wall 193.7 -> 164.5 s). The prediction I shipped with the arm -- slab is a
threshold below ~26 GiB, so 1,600 MiB buys ~0 -- is falsified by the measurement
it asked for. Two slab points inside noise of EACH OTHER (22,208 / 24,000) are
not a plateau. The cliff is above 25,600: 26,071 measured 4,379 t/s.

Ceiling re-derived from the engine's own budget table (card 32,624 MiB, pair at
29,120 settling 2,058 MiB free against its 209.19 MiB floor => 30,969 claimable).
The 29,120 bound was my 1 GiB \`keep free\` reserve; the seam now derives the
number, so the next resize starts from the engine's arithmetic.

Ledger for the record: slab 16,235 -> 17,548 slots while the pinned host pool
hands the same experts back (8,893 -> 7,580) -- the transfer is the mechanism,
the pinned pool is the slow path. Capacity contract live-verified at 3 slots:
3 x 199,984-token prompts accepted (599,952 tokens), fourth refused as too
large to ever fit.

Probe law learned: the oversized-prompt refusal names the prompt's token count
BEFORE prefill (free calibration) and fires on prompt tokens ALONE, not on
prompt + max_tokens; the window is ~514,286 chars at 2.5714 chars/token.
This commit is contained in:
Frank committed 2026-10-13 20:33:02 -07:00
1 parent 09a0251d2d
commit 2581b8d4b6
5 files changed
+153 -42

No files matched your search

+27 -18
View File
@@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Model | Port | | Target | Service | Runtime | Model | Port |
|--------|---------|---------|-------|------| |--------|---------|---------|-------|------|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **44 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 22208 --vram-kv-mib 6912` = the 29,120 MiB/card ceiling), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (pool 6,912 MiB ≈ 843k tokens — the only dtype that funds four slots, since bf16 at 4 × 200,000 costs 11,679 MiB of pool and would cut the slab to 17,441), **MTP spec decode depth 2**, `--max-num-seqs 4` (the capacity contract is 4 × 200,000; 4 × (1+2) = 12 all-reduce rows, inside the 25-row wht6 exactness bound), `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback); 4-slot fp8-KV shape 2026-10-13, the 2-slot bf16 shape (22208/6912, `--max-num-seqs 2`) is the documented rollback. | 7770 | | Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **44 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the **30,969 MiB** the engine's own budget table makes claimable: card 32,624 MiB, settle floor 209 MiB), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (pool 5,120 MiB = the 3-slot contract + 1.3 % — the only dtype that funds 3 × 200,000 without gutting the slab, since bf16 bills 13.8 KiB/token), **MTP spec decode depth 2**, `--max-num-seqs 3` (the capacity contract is 3 × 200,000; 3 × (1+2) = 9 all-reduce rows, inside the 25-row wht6 exactness bound), `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8); prefill is **slab-limited and pays for it** — 22,208 → 3,397, 24,000 → 3,290, 25,600 → **3,584** t/s at 43.5k single-stream, 3-way 197k aggregate 3,051 → **3,589** t/s, with a cliff at ~26 GiB (26,071 → 4,379). Cutover 2026-10-08 from fp8-native (payload on disk = rollback); 3-slot fp8-KV shape 2026-10-13, re-cut twice in the slab's favour (22208/6912 at 4 slots → 24000/5120 → 25600/5120); the 4-slot shape (22208/6912, `--max-num-seqs 4`) is the documented rollback. | 7770 |
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770| | Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 | | Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
@@ -525,26 +525,35 @@ the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar
Rollback: revert the swap commit — the fp8-native `.rad` Rollback: revert the swap commit — the fp8-native `.rad`
(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk. (`qwen4exp-fp8-native.rad`) and every prior payload remain on disk.
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 4`, **Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
`--max-model-len 200000` (the harness contextWindow must quote this), `--max-model-len 200000` (the harness contextWindow must quote this),
`--placement expert_tiered --host-pool-mib 44928` + **explicit VRAM `--placement expert_tiered --host-pool-mib 44928` + **explicit VRAM
partition `--vram-weights-mib 22208 --vram-kv-mib 6912`** (the vram-pair partition `--vram-weights-mib 25600 --vram-kv-mib 5120`** (the vram-pair
replaced `--expert-vs-cache-ratio`; the two are ONE budget — replaced `--expert-vs-cache-ratio`; the two are ONE budget —
`weights + kv ≤ 29,120 MiB`, from the measured card 31,856 minus 1,710 `weights + kv ≤ 30,969 MiB`, derived from the engine's own budget table: card
consumed outside both budgets minus 1,024 to keep free), `--kv-cache-dtype` 32,624 MiB, the pair at 29,120 settling 2,058 MiB free against the engine's
**`fp8` — quantized KV, restored 2026-10-13 because it is the only dtype that 209.19 MiB floor (`--gpu-headroom-mib 96` + 113.19 static), so 1,849 MiB is
funds four slots at this window**, `--num-speculative-tokens 2`, claimable and the old 29,120 bound was my hand-picked 1 GiB reserve, not a
`--max-num-batched-tokens 2048`, prefix cache 8 GiB pinned host + 64 GiB hardware wall), `--kv-cache-dtype` **`fp8` — quantized KV, restored 2026-10-13
disk at `/home/dank/radiance-state`, **`--ngram-placement ram`**, Froggeric because it is the only dtype that funds the slot count at this window**,
v22.5 chat template vendored (`scripts/qwen38-froggeric-v22.5.jinja`, `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`, prefix cache
`--override-chat-template`). Boot ledger (2026-10-13): `kv cache 6.75 GiB … 8 GiB pinned host + 64 GiB disk at `/home/dank/radiance-state`,
backs 4 sequences all at 200000 tokens at once, which is the worst case` — **`--ngram-placement ram`**, Froggeric v22.5 chat template vendored
`kv_attn` page 26.00 KiB (fp8) at 843,276 tokens (the sizing law predicted (`scripts/qwen38-froggeric-v22.5.jinja`, `--override-chat-template`). Boot
843,566), `kv_qsa_bkey` 3.25 KiB at the same token count, `kv_gdn_state` 54 MiB ledger (2026-10-13, pair 25600/5120): `kv cache 5.00 GiB … backs 3 sequences
× 8 slots (4 sequences × 2 copies) = 432 MiB; the sampler prints **12 rows** all at 200000 tokens at once, which is the worst case` — `kv_attn` page
(4 × (1+2), inside the 25-row `wht6` exactness bound); weights filled to 26.00 KiB (fp8) at 607,740 tokens (the sizing law predicted 608,027),
147 KiB of slack, cards settled ~2 GiB free, and the KV pool lends 5.87 GiB to `kv_qsa_bkey` 3.25 KiB at the same token count, `kv_gdn_state` 54 MiB × 6 slots
the expert slab (the fp8-era 3,543 MiB pool lent 2.69 GiB). (3 sequences × 2 copies) = 324 MiB; the sampler prints **9 rows**
(3 × (1+2), inside the 25-row `wht6` exactness bound); the slab filled to
563 KiB of slack with **17,548 slots** (16,235 at 24,000, 14,753 at 22,208 —
every 1,792 MiB buys ~1,300 resident experts, and the pinned host pool hands
the same experts back: 8,893 → 7,580 slots); `unclaimed 477.30 MiB`, cards
settled ~336 MiB free against the engine's 209.19 MiB floor, KV lending
4.22 GiB to the expert cache. Prefill tracks this slab: the 24,000 → 25,600
step measured +8.9 % single-stream (3,290 → 3,584 t/s at 43.5k) and +17.6 %
concurrent (3-way 197k aggregate), so the pinned host pool is the slow path and
every MiB moved from RAM onto the card is throughput.
Shape notes carried from the fp8-native era and unchanged: wht6 gains, Shape notes carried from the fp8-native era and unchanged: wht6 gains,
MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
+13 -6
View File
@@ -194,12 +194,19 @@ FRANK_ARCHON_GPU_HEADROOM_MIB=96
# contract), so the boot ledger's `backs N sequences all at 200000 tokens at # contract), so the boot ledger's `backs N sequences all at 200000 tokens at
# once` line is the referee for THIS pair and the concurrent probe is its # once` line is the referee for THIS pair and the concurrent probe is its
# regression test. # regression test.
# FALSIFIABLE CLAIM for the 25,600 arm: today measured slab 22,208 -> 3,397 # VERDICT on the 25,600 arm (measured 2026-10-13, scripts/probe-frank-archon-kv.py):
# t/s and 24,000 -> 3,290 t/s at 43.5k single-stream, so below ~26 GiB the slab # the threshold claim is FALSIFIED — the slab is a SLOPE at this end. A-cold
# is a THRESHOLD, not a slope. Expected: +1,600 resident expert slots, ~0 # 43.5k single-stream: 22,208 -> 3,397, 24,000 -> 3,290, 25,600 -> 3,584 t/s;
# prefill. The point that DOES move prefill is 26,071 (the 2026-10-11 fp8 arm, # the 3-way 197k arm's aggregate prefill 3,051 -> 3,589 t/s (wall
# 4,379 t/s), fundable only by dropping to two slots. Prefill moving more than # 193.7 -> 164.5 s). So +1,600 MiB of slab = +8.9 % single-stream and
# +-3 % on this arm falsifies the threshold reading. # +17.6 % concurrent, both outside the +-3 % noise floor. The 22,208/24,000
# pair landed inside noise of EACH OTHER, which is what made it read as a
# plateau; two points are not a curve.
# The cliff that remains is between 25,600 (3,584) and 26,071 (4,379, the
# 2026-10-11 fp8 control at seqs 3 / pool 3,543) — +22 % for +471 MiB. At THREE
# slots that is only reachable by breaking the ceiling (26,112 + 5,120 = 31,232
# vs 30,969, ~214 MiB above the engine's floor); at TWO slots it is free:
# pool 3,072 + slab 27,897 = exactly the ceiling, slab above the cliff.
FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600 FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600
FRANK_ARCHON_VRAM_KV_MIB=5120 FRANK_ARCHON_VRAM_KV_MIB=5120
# KV cache QUANTIZED (fp8). This engine's flag takes ONLY bf16|fp8 — there is # KV cache QUANTIZED (fp8). This engine's flag takes ONLY bf16|fp8 — there is
@@ -145,11 +145,18 @@ dtype`) so a future resize cannot silently under-provision the pool.
### Why the weights slab pays — and why that is not pure loss ### Why the weights slab pays — and why that is not pure loss
The two VRAM budgets are one budget: card 31,856 MiB, minus the 1,710 MiB a The two VRAM budgets are one budget, and the ceiling is the engine's own — not a
boot consumes outside both (driver context, the 990 MiB activation arena, hand-picked reserve. Card 32,624 MiB, and the table that prints at the pair
hardware queues, headroom, step staging), minus 1,024 MiB kept free ⇒ 24,000/5,120 says `3.42 GiB is neither budgeted nor accounted — the driver's
**`weights + kv ≤ 29,120 MiB`**. Paying for a 6,912 MiB pool therefore cuts the context, the activation arena and --gpu-headroom-mib come out of it; anything
resident slab 26,071 → 22,208 MiB (−14.8 %), and the evicted experts spill past those is throughput on the floor`, with `card 0 settled at 2.01 GiB free
against 209.19 MiB (the 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the
static region)`. Claimable reserve above the ENGINE's floor:
2,058 − 209 = 1,849 MiB ⇒ **`weights + kv ≤ 30,969 MiB`**. (The 29,120 bound
written on 2026-10-11 was my own 1 GiB "keep free" reserve, measured off a card
figure of 31,856 MiB; the engine never asked for that reserve and the seam now
derives the number from its table.) Paying for a 6,912 MiB pool therefore cuts
the resident slab 26,071 → 22,208 MiB (−14.8 %), and the evicted experts spill
into the pinned host pool (40,960 → 44,928 MiB per rank; ledger: pinned pool into the pinned host pool (40,960 → 44,928 MiB per rank; ledger: pinned pool
slots 7,190 → 10,375). The pinned-RAM ledger rises to ≈147 GiB, so the TTM slots 7,190 → 10,375). The pinned-RAM ledger rises to ≈147 GiB, so the TTM
floor moved 140,000 → 150,000 MiB (the kernel cap is 204,800). floor moved 140,000 → 150,000 MiB (the kernel cap is 204,800).
@@ -303,7 +310,7 @@ knob ends at the target and the start passes; a read-only knob leaves the cap
untouched and the start refuses by name; an absent `pages_max` is skipped, never untouched and the start refuses by name; an absent `pages_max` is skipped, never
read as zero (a zero reading would refuse a healthy host). read as zero (a zero reading would refuse a healthy host).
### The shape: 4 slots × 200,000, fp8 KV ### The shape: 4 slots × 200,000, fp8 KV (same day, superseded by the re-cuts below)
Answers the open call of 2026-10-11 from the other side: Frank asked for **four** Answers the open call of 2026-10-11 from the other side: Frank asked for **four**
concurrent 200,000-token conversations with quantized KV. concurrent 200,000-token conversations with quantized KV.
@@ -311,7 +318,7 @@ concurrent 200,000-token conversations with quantized KV.
fp8 is not a preference at four slots, it is the only dtype that affords them. fp8 is not a preference at four slots, it is the only dtype that affords them.
The pool must back `seqs × max-model-len` (the engine's hard limit), so the The pool must back `seqs × max-model-len` (the engine's hard limit), so the
fourth slot is bought in KV, out of the one per-card budget fourth slot is bought in KV, out of the one per-card budget
(`weights + kv ≤ 29,120 MiB`): (`weights + kv ≤ 29,120 MiB`, the bound as it was understood then):
| dtype | pool for 4 × 200,000 | weights slab left | | dtype | pool for 4 × 200,000 | weights slab left |
|---|---|---| |---|---|---|
@@ -345,7 +352,7 @@ cold / ~3.3k warm; decode sits in the 94–118 t/s band with draft acceptance
~0.73–0.75. The 4,379 t/s fp8 prefill of the 2026-10-08 control belongs to the ~0.73–0.75. The 4,379 t/s fp8 prefill of the 2026-10-08 control belongs to the
26,071 MiB slab and does not come back by quantizing KV. 26,071 MiB slab and does not come back by quantizing KV.
### Boot ledger (2026-10-13 19:21, four slots, fp8) ### Boot ledger (2026-10-13 19:21, four slots, fp8 — superseded same evening)
The sizing law's prediction, checked against the engine's own arithmetic: The sizing law's prediction, checked against the engine's own arithmetic:
predicted 843,566 tokens, ledger **843,276** (0.03 % block rounding). predicted 843,566 tokens, ledger **843,276** (0.03 % block rounding).
@@ -370,6 +377,50 @@ Two ledger lines that confirm the derived consequences: the sampler prints
`wht6` is bit-exact), and the prefix/tier arithmetic runs on the fp8 entry size `wht6` is bit-exact), and the prefix/tier arithmetic runs on the fp8 entry size
(58.50 KiB, half of the bf16 entry). (58.50 KiB, half of the bf16 entry).
### The same-day re-cuts: 4 → 3 slots, then the slab takes the card's slack
Frank's call, twice in one evening. Both are one-env-pair changes, and both were
read off the boot ledger rather than assumed.
**4 → 3 slots (22208/6912 → 24000/5120).** Three slots need
3 × 200,000 at fp8 = 4,285 + 330 + 448 = **5,063 MiB** of pool; the 6,912 pool
was backing 858,969 tokens — 43 % more window than the contract asks for.
Cutting it to 5,120 (predicts 608,027; ledger **607,740**, 0.05 %) hands
1,792 MiB to the slab and drops the all-reduce to 9 rows.
**24,000 → 25,600 (the card's settling slack).** The 24,000 boot printed
`card 0 settled at 2.01 GiB free against 209.19 MiB (the 96.00 MiB
--gpu-headroom-mib and 113.19 MiB kept for the static region)` — 1,849 MiB above
the ENGINE's own floor, with the budget table calling the surplus `throughput on
the floor`. Claiming 1,600 MiB of it puts the pair at 30,720 and the ledger at
`unclaimed 477.30 MiB` (cards ~336 MiB free, floor 209.19).
| pair | slab slots (rank 0) | pinned pool slots | A-cold 43.5k | concurrent 197k |
|---|---|---|---|---|
| 22208/6912, seqs 4 | 14,753 | 10,375 | 3,397 t/s | 3,307 t/s (4-way) |
| 24000/5120, seqs 3 | 16,235 | 8,893 | 3,290 t/s | 3,051 t/s (3-way) |
| **25600/5120, seqs 3** | **17,548** | 7,580 | **3,584 t/s** | **3,589 t/s** (3-way) |
| 26071/3543, seqs 3 (2026-10-11 control) | — | 7,190 | 4,379 t/s | — |
The claim shipped with the 25,600 arm — that the slab is a THRESHOLD below
~26 GiB so the extra 1.6 GiB buys ~0 — is **falsified by its own measurement**:
+8.9 % single-stream, +17.6 % concurrent, both outside the ±3 % noise floor. Two
slab points that land inside noise of *each other* (22,208 vs 24,000) are not a
plateau; the pinned host pool is the slow path, so every MiB moved from RAM onto
the card is throughput (the ledger shows the transfer directly: slab
+1,313 slots, pinned pool −1,313).
The cliff is real, it just sits above 25,600: 26,071 measured 4,379 t/s, +22 %
for +471 MiB. At three slots that is out of reach (26,112 + 5,120 = 31,232 > the
30,969 ceiling). At **two** slots the pool falls to 3,072 and the slab reaches
**27,897** — the ceiling exactly, slab above the cliff. That arm is untested and
it is a capacity decision, not a tuning one.
Capacity contract, live-verified on the 3-slot shape: three concurrent prompts
of **199,984 tokens** each (599,952 tokens) were accepted and completed on
`/v1/completions`; a fourth full-height request is refused as "too large to ever
fit" — radiance never truncates.
### Recovery evidence ### Recovery evidence
```text ```text
+19 -10
View File
@@ -6,8 +6,9 @@ driving `podman run` directly, root systemd unit, :7770):
`ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4** `ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4**
(`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s (`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
(explicit `--vram-weights-mib 22208 --vram-kv-mib 6912`), the 47.7 GiB PLE table (explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
pinned to host RAM, **fp8 KV at 4 × 200,000**, MTP spec decode depth 2, 30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005 froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005
and frank-archon-vllm-shapeG-20261003), and frank-archon-vllm-shapeG-20261003),
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2 and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
@@ -155,14 +156,22 @@ frank-zealot, port 7771]
- [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**: - [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**:
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container); 1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
the FP8-native payload remains the rollback. **2026-10-13: 4 x 200,000 with the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
fp8 KV** — `--max-num-seqs 4` + `--kv-cache-dtype fp8` in the same 6912 MiB pair re-cut twice** — `--kv-cache-dtype fp8` first at
pool (843,566 tokens), slab 22208 and host pool 44928 untouched. fp8 is what `--max-num-seqs 4`/6912 MiB pool (843,566 tokens, slab 22208), then **3 slots
funds the fourth slot: the pool must back seqs x window, and at bf16 four with the pool cut to the contract (5120 MiB, 608,027 tokens) and the slab
slots cost 11,679 MiB, which would cut the resident slab to 17,441 (-33 %). raised 22208 → 24000 → 25600**, the last 1,600 MiB claimed out of the card's
The bf16 arm (2 x 200,000, 462,752 tokens, prefill -29 %, acceptance settling slack: the engine's ceiling is **30,969 MiB**, not the 29,120 my
0.661 → 0.740) is the documented rollback. **2026-10-13 also: the pinned-ledger 1 GiB reserve invented. fp8 is what funds the slots — the pool must back
preflight now raises `ttm.pages_limit` itself** — the 2026-10-12 power failure `seqs × window`, and at bf16 four slots cost 11,679 MiB (slab 17,441, −33 %).
Measured on the slab: A-cold 43.5k prefill 22208 → 3,397, 24000 → 3,290,
**25600 → 3,584 t/s**; 3-way 197k aggregate 3,051 → **3,589 t/s**. A shipped
prediction that the slab was a threshold below 26 GiB died on its own
measurement — it is a slope, and 26,071 (4,379 t/s) is the cliff. The bf16
arm (2 × 200,000, 462,752 tokens, prefill −29 %, acceptance 0.661 → 0.740)
and the 4-slot fp8 shape are the documented rollbacks.
**2026-10-13 also: the pinned-ledger preflight now raises `ttm.pages_limit`
itself** — the 2026-10-12 power failure
reset the knob to its default (125,988 MiB) and the endpoint refused to boot reset the knob to its default (125,988 MiB) and the endpoint refused to boot
6,919 times in ~20 h; `scripts/ttm-preflight.sh` is the repair. 6,919 times in ~20 h; `scripts/ttm-preflight.sh` is the repair.
- [[frank-archon-radiance-qwen38-native-fp8]] — **RETIRED 2026-10-08 (the - [[frank-archon-radiance-qwen38-native-fp8]] — **RETIRED 2026-10-08 (the
+35
View File
@@ -32,6 +32,41 @@
aggregate prefill, no preemption; prefix cache verified (repeat answered in aggregate prefill, no preemption; prefix cache verified (repeat answered in
1.0 s, `cache_n=43008`). Playwright e2e 32 passed + 1 flaky (the known Plex 1.0 s, `cache_n=43008`). Playwright e2e 32 passed + 1 flaky (the known Plex
homepage retry); archon bats seam 21/21 including three new ttm-gate tests. homepage retry); archon bats seam 21/21 including three new ttm-gate tests.
- **Re-cut twice the same evening (Frank's calls), each read off the ledger.**
4 → 3 slots, pool 6,912 → 5,120: three slots need 5,063 MiB and the 6,912 pool
was funding 858,969 tokens — 43 % over the contract. Then slab
24,000 → 25,600, claiming 1,600 of the 1,849 MiB the engine leaves above its
OWN settle floor (`card 0 settled at 2.01 GiB free against 209.19 MiB (the
96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)`): the
real ceiling is 30,969 MiB, not the 29,120 my 1 GiB "keep free" reserve
invented on 2026-10-11. The pair now spends 30,720 and settles
`unclaimed 477.30 MiB` (cards ~336 MiB free).
- **The prediction shipped with that arm was falsified by its own measurement,
which is the whole point of banking the A/B.** Claim: the slab is a THRESHOLD
below ~26 GiB (22,208 → 3,397 and 24,000 → 3,290 t/s are inside noise of each
other), so 1.6 GiB buys ~0. Measured at 25,600: **3,584 t/s** single-stream
cold at 43.5k (+8.9 %) and **3,589 t/s** 3-way aggregate at 197k (wall
193.7 → 164.5 s, +17.6 %) — slab 16,235 → 17,548 slots while the pinned host
pool hands the same experts back (8,893 → 7,580). The slab is a SLOPE; two
points inside noise of each other are not a plateau. The cliff is above
25,600: 26,071 measured 4,379 t/s (+22 % for +471 MiB), unreachable at three
slots (26,112 + 5,120 > the ceiling) and free at two.
- **Capacity verified live at 3 slots:** three concurrent 199,984-token prompts
accepted and completed (599,952 tokens resident), ledger
`backs 3 sequences all at 200000 tokens at once` at 607,740 tokens against the
law's 608,027. A fourth full-height request is refused as too large to ever
fit — radiance never truncates. Probe law: the oversized-prompt counter is
FREE (the refusal fires when PROMPT tokens alone exceed the window —
`prompt: 202,777 prompt tokens does not fit the 200000 token context`) and the
window is ~514,286 characters at the measured 2.5714 chars/token, so a
calibration probe below that pays a real prefill instead of reporting.
- **Open call for Frank:** two slots (pool 3,072 + slab 27,897 = the ceiling
exactly) crosses the cliff for the +22 %, and costs the third concurrent 200k
conversation. Untested arm; capacity decision, not tuning.
- Seam: the pair ceiling is derived from the engine's budget table (30,969) and
owns that arithmetic alone; the KV-pool contract (seqs × window at the dtype's
measured rate + 110 MiB a sequence + 448 MiB of prefix checkpoints) keeps its
own test. 371/371 tests and e2e 33/33 on both deploys.
- **Power-failure casualties, NOT fixed (outside the archon window):** - **Power-failure casualties, NOT fixed (outside the archon window):**
`mnt-md0.mount` failed at boot, so `/mnt/md0` is an empty mountpoint (the `mnt-md0.mount` failed at boot, so `/mnt/md0` is an empty mountpoint (the
raid6 array itself is healthy, 7/7; fstab has `nofail` so boot continued) — raid6 array itself is healthy, 7/7; fstab has `nofail` so boot continued) —