docs(frank-archon): the slab is a slope, and the ceiling was mine not the engine's

Measured A/B on the shipped pair 25600/5120: A-cold 43.5k prefill 3,290 ->
3,584 t/s (+8.9 %) and the 3-way 197k aggregate 3,051 -> 3,589 t/s (+17.6 %,
wall 193.7 -> 164.5 s). The prediction I shipped with the arm -- slab is a
threshold below ~26 GiB, so 1,600 MiB buys ~0 -- is falsified by the measurement
it asked for. Two slab points inside noise of EACH OTHER (22,208 / 24,000) are
not a plateau. The cliff is above 25,600: 26,071 measured 4,379 t/s.

Ceiling re-derived from the engine's own budget table (card 32,624 MiB, pair at
29,120 settling 2,058 MiB free against its 209.19 MiB floor => 30,969 claimable).
The 29,120 bound was my 1 GiB \`keep free\` reserve; the seam now derives the
number, so the next resize starts from the engine's arithmetic.

Ledger for the record: slab 16,235 -> 17,548 slots while the pinned host pool
hands the same experts back (8,893 -> 7,580) -- the transfer is the mechanism,
the pinned pool is the slow path. Capacity contract live-verified at 3 slots:
3 x 199,984-token prompts accepted (599,952 tokens), fourth refused as too
large to ever fit.

Probe law learned: the oversized-prompt refusal names the prompt's token count
BEFORE prefill (free calibration) and fires on prompt tokens ALONE, not on
prompt + max_tokens; the window is ~514,286 chars at 2.5714 chars/token.
This commit is contained in:
Frank
2026-10-13 20:33:02 -07:00
parent 09a0251d2d
commit 2581b8d4b6
5 changed files with 153 additions and 42 deletions
+19 -10
View File
@@ -6,8 +6,9 @@ driving `podman run` directly, root systemd unit, :7770):
`ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4**
(`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
(explicit `--vram-weights-mib 22208 --vram-kv-mib 6912`), the 47.7 GiB PLE table
pinned to host RAM, **fp8 KV at 4 × 200,000**, MTP spec decode depth 2,
(explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005
and frank-archon-vllm-shapeG-20261003),
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
@@ -155,14 +156,22 @@ frank-zealot, port 7771]
- [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**:
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
the FP8-native payload remains the rollback. **2026-10-13: 4 x 200,000 with
fp8 KV** — `--max-num-seqs 4` + `--kv-cache-dtype fp8` in the same 6912 MiB
pool (843,566 tokens), slab 22208 and host pool 44928 untouched. fp8 is what
funds the fourth slot: the pool must back seqs x window, and at bf16 four
slots cost 11,679 MiB, which would cut the resident slab to 17,441 (-33 %).
The bf16 arm (2 x 200,000, 462,752 tokens, prefill -29 %, acceptance
0.661 → 0.740) is the documented rollback. **2026-10-13 also: the pinned-ledger
preflight now raises `ttm.pages_limit` itself** — the 2026-10-12 power failure
the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
pair re-cut twice** — `--kv-cache-dtype fp8` first at
`--max-num-seqs 4`/6912 MiB pool (843,566 tokens, slab 22208), then **3 slots
with the pool cut to the contract (5120 MiB, 608,027 tokens) and the slab
raised 22208 → 24000 → 25600**, the last 1,600 MiB claimed out of the card's
settling slack: the engine's ceiling is **30,969 MiB**, not the 29,120 my
1 GiB reserve invented. fp8 is what funds the slots — the pool must back
`seqs × window`, and at bf16 four slots cost 11,679 MiB (slab 17,441, −33 %).
Measured on the slab: A-cold 43.5k prefill 22208 → 3,397, 24000 → 3,290,
**25600 → 3,584 t/s**; 3-way 197k aggregate 3,051 → **3,589 t/s**. A shipped
prediction that the slab was a threshold below 26 GiB died on its own
measurement — it is a slope, and 26,071 (4,379 t/s) is the cliff. The bf16
arm (2 × 200,000, 462,752 tokens, prefill −29 %, acceptance 0.661 → 0.740)
and the 4-slot fp8 shape are the documented rollbacks.
**2026-10-13 also: the pinned-ledger preflight now raises `ttm.pages_limit`
itself** — the 2026-10-12 power failure
reset the knob to its default (125,988 MiB) and the endpoint refused to boot
6,919 times in ~20 h; `scripts/ttm-preflight.sh` is the repair.
- [[frank-archon-radiance-qwen38-native-fp8]] — **RETIRED 2026-10-08 (the