docs(perf): destroy §9's vendor absolutes, keep the ratios

The t/s ladder and the TTFT delta triple measure a configuration neither
deployed nor compared; per the purpose test they are dead weight. What survives
is the knob pricing (halving the shared chunk costs roughly a fifth of prefill
throughput), the long-prompt-only shape of the span win, and the method. Run-on
now names the era the census was taken in.
This commit is contained in:
Frank committed 2026-10-19 14:55:43 -07:00
1 parent d9b82b372e
commit 7eed636bd1
1 file changed
+15 -15
@@ -251,21 +251,20 @@ Knobs, in the order worth trying them (radiance GUIDE §4.5):
the prefill stager's smallest pass — the startup line names it, and below it every the prefill stager's smallest pass — the startup line names it, and below it every
routed layer reads its pooled experts synchronously, which costs the prefill far routed layer reads its pooled experts synchronously, which costs the prefill far
more than sharing saves. Counts as `engine_prefill_chunks_capped_total`. more than sharing saves. Counts as `engine_prefill_chunks_capped_total`.
- `--max-num-batched-tokens` — the blunt rung, and the only one older pins have. The - `--max-num-batched-tokens` — the blunt rung, and the only one older pins have. The vendor's own
vendor's curve on a 48-layer MoE at 200k on two cards: chunk 512 → 1750 t/s, 1024 curve prices **halving the shared chunk at roughly a fifth of prefill throughput** (a 48-layer MoE
→ 2200, 2048 → 2706 (a 185k prompt: 138 s against 91 s), so halving the stall at 200k on two cards — knob pricing, not this endpoint's number, so treat it as the order of
costs ~19 % of prefill. Raising it past the interval buys nothing and loses expert magnitude a chunk decision costs). Raising it past the checkpoint interval buys nothing and loses
residency to the arena. expert residency to the arena.
- `--prefill-span N` — the inverse trade: a *lone* prefill carries N intervals a step - `--prefill-span N` — the inverse trade: a *lone* prefill carries N intervals a step while every
while every shared step is planned at a smaller shared budget, which on an shared step is planned at a smaller shared budget, which on an expert-tiered box lends that arena
expert-tiered box lends that arena back to the expert slab. Vendor TP2 back to the expert slab. The vendor's own measurement says the win is **long-prompt-only**: nothing
expert-tiered TTFT deltas vs a plain 2048 budget: −6.1 % at 2k, +2.3 % at 18k, at short prompts, a large TTFT gain at the far end of the window.
+49.1 % at 250k — long prompts only.
Two traps. **The arena/slab coupling makes a shared step doubly expensive on an Two traps. **The arena/slab coupling makes a shared step doubly expensive on an
expert-tiered shape**: the full-plan arena takes VRAM the resident expert slab would expert-tiered shape**: the full-plan arena takes VRAM the resident expert slab would have had, so the
have had (320 MiB/card at TP2 for a 4096 budget), so the decode rows inside that step decode rows inside that step also read experts over the link — a smaller shared budget helps twice.
also read experts over the link — a smaller shared budget helps twice. And **re-check And **re-check
`prefix_hit_rate` / `prefix_evictions` after any chunk change**: the arena and the KV `prefix_hit_rate` / `prefix_evictions` after any chunk change**: the arena and the KV
pool are the same memory, and the failure looks like turn two taking as long as turn pool are the same memory, and the failure looks like turn two taking as long as turn
one. one.
@@ -276,8 +275,9 @@ Availability is a version question, not a config question: `--busy-prefill-chunk
tree, and check the source clone's depth before dating a flag from `git log -S` tree, and check the source clone's depth before dating a flag from `git log -S`
(a `--depth 1` clone reports its own tip for every search). (a `--depth 1` clone reports its own tip for every search).
Run on: [[frank-archon]] (radiance 1.0.13, TP2 `expert_tiered`, budget 2048 = interval Run on: [[frank-archon]] — the census was taken on its 3-slot radiance 1.0.13 era (budget 2048 =
2048) — the measured census and the verdict for this fleet's shape are that page's. interval 2048); the live 1.8.1 shape carries the two chunk knobs. The measured census and the verdict
for this fleet's shape are that page's.
## Related ## Related