docs(frank-archon): the slab is a slope, and the ceiling was mine not the engine's
Measured A/B on the shipped pair 25600/5120: A-cold 43.5k prefill 3,290 -> 3,584 t/s (+8.9 %) and the 3-way 197k aggregate 3,051 -> 3,589 t/s (+17.6 %, wall 193.7 -> 164.5 s). The prediction I shipped with the arm -- slab is a threshold below ~26 GiB, so 1,600 MiB buys ~0 -- is falsified by the measurement it asked for. Two slab points inside noise of EACH OTHER (22,208 / 24,000) are not a plateau. The cliff is above 25,600: 26,071 measured 4,379 t/s. Ceiling re-derived from the engine's own budget table (card 32,624 MiB, pair at 29,120 settling 2,058 MiB free against its 209.19 MiB floor => 30,969 claimable). The 29,120 bound was my 1 GiB \`keep free\` reserve; the seam now derives the number, so the next resize starts from the engine's arithmetic. Ledger for the record: slab 16,235 -> 17,548 slots while the pinned host pool hands the same experts back (8,893 -> 7,580) -- the transfer is the mechanism, the pinned pool is the slow path. Capacity contract live-verified at 3 slots: 3 x 199,984-token prompts accepted (599,952 tokens), fourth refused as too large to ever fit. Probe law learned: the oversized-prompt refusal names the prompt's token count BEFORE prefill (free calibration) and fires on prompt tokens ALONE, not on prompt + max_tokens; the window is ~514,286 chars at 2.5714 chars/token.
This commit is contained in:
@@ -6,8 +6,9 @@ driving `podman run` directly, root systemd unit, :7770):
|
||||
`ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4**
|
||||
(`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s
|
||||
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
|
||||
(explicit `--vram-weights-mib 22208 --vram-kv-mib 6912`), the 47.7 GiB PLE table
|
||||
pinned to host RAM, **fp8 KV at 4 × 200,000**, MTP spec decode depth 2,
|
||||
(explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
|
||||
30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
|
||||
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
|
||||
froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005
|
||||
and frank-archon-vllm-shapeG-20261003),
|
||||
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
||||
@@ -155,14 +156,22 @@ frank-zealot, port 7771]
|
||||
- [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**:
|
||||
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
|
||||
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
|
||||
the FP8-native payload remains the rollback. **2026-10-13: 4 x 200,000 with
|
||||
fp8 KV** — `--max-num-seqs 4` + `--kv-cache-dtype fp8` in the same 6912 MiB
|
||||
pool (843,566 tokens), slab 22208 and host pool 44928 untouched. fp8 is what
|
||||
funds the fourth slot: the pool must back seqs x window, and at bf16 four
|
||||
slots cost 11,679 MiB, which would cut the resident slab to 17,441 (-33 %).
|
||||
The bf16 arm (2 x 200,000, 462,752 tokens, prefill -29 %, acceptance
|
||||
0.661 → 0.740) is the documented rollback. **2026-10-13 also: the pinned-ledger
|
||||
preflight now raises `ttm.pages_limit` itself** — the 2026-10-12 power failure
|
||||
the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
|
||||
pair re-cut twice** — `--kv-cache-dtype fp8` first at
|
||||
`--max-num-seqs 4`/6912 MiB pool (843,566 tokens, slab 22208), then **3 slots
|
||||
with the pool cut to the contract (5120 MiB, 608,027 tokens) and the slab
|
||||
raised 22208 → 24000 → 25600**, the last 1,600 MiB claimed out of the card's
|
||||
settling slack: the engine's ceiling is **30,969 MiB**, not the 29,120 my
|
||||
1 GiB reserve invented. fp8 is what funds the slots — the pool must back
|
||||
`seqs × window`, and at bf16 four slots cost 11,679 MiB (slab 17,441, −33 %).
|
||||
Measured on the slab: A-cold 43.5k prefill 22208 → 3,397, 24000 → 3,290,
|
||||
**25600 → 3,584 t/s**; 3-way 197k aggregate 3,051 → **3,589 t/s**. A shipped
|
||||
prediction that the slab was a threshold below 26 GiB died on its own
|
||||
measurement — it is a slope, and 26,071 (4,379 t/s) is the cliff. The bf16
|
||||
arm (2 × 200,000, 462,752 tokens, prefill −29 %, acceptance 0.661 → 0.740)
|
||||
and the 4-slot fp8 shape are the documented rollbacks.
|
||||
**2026-10-13 also: the pinned-ledger preflight now raises `ttm.pages_limit`
|
||||
itself** — the 2026-10-12 power failure
|
||||
reset the knob to its default (125,988 MiB) and the endpoint refused to boot
|
||||
6,919 times in ~20 h; `scripts/ttm-preflight.sh` is the repair.
|
||||
- [[frank-archon-radiance-qwen38-native-fp8]] — **RETIRED 2026-10-08 (the
|
||||
|
||||
Reference in New Issue
Block a user