docs(perf): destroy §9's vendor absolutes, keep the ratios
The t/s ladder and the TTFT delta triple measure a configuration neither deployed nor compared; per the purpose test they are dead weight. What survives is the knob pricing (halving the shared chunk costs roughly a fifth of prefill throughput), the long-prompt-only shape of the span win, and the method. Run-on now names the era the census was taken in.
This commit is contained in:
@@ -251,21 +251,20 @@ Knobs, in the order worth trying them (radiance GUIDE §4.5):
|
||||
the prefill stager's smallest pass — the startup line names it, and below it every
|
||||
routed layer reads its pooled experts synchronously, which costs the prefill far
|
||||
more than sharing saves. Counts as `engine_prefill_chunks_capped_total`.
|
||||
- `--max-num-batched-tokens` — the blunt rung, and the only one older pins have. The
|
||||
vendor's curve on a 48-layer MoE at 200k on two cards: chunk 512 → 1750 t/s, 1024
|
||||
→ 2200, 2048 → 2706 (a 185k prompt: 138 s against 91 s), so halving the stall
|
||||
costs ~19 % of prefill. Raising it past the interval buys nothing and loses expert
|
||||
residency to the arena.
|
||||
- `--prefill-span N` — the inverse trade: a *lone* prefill carries N intervals a step
|
||||
while every shared step is planned at a smaller shared budget, which on an
|
||||
expert-tiered box lends that arena back to the expert slab. Vendor TP2
|
||||
expert-tiered TTFT deltas vs a plain 2048 budget: −6.1 % at 2k, +2.3 % at 18k,
|
||||
+49.1 % at 250k — long prompts only.
|
||||
- `--max-num-batched-tokens` — the blunt rung, and the only one older pins have. The vendor's own
|
||||
curve prices **halving the shared chunk at roughly a fifth of prefill throughput** (a 48-layer MoE
|
||||
at 200k on two cards — knob pricing, not this endpoint's number, so treat it as the order of
|
||||
magnitude a chunk decision costs). Raising it past the checkpoint interval buys nothing and loses
|
||||
expert residency to the arena.
|
||||
- `--prefill-span N` — the inverse trade: a *lone* prefill carries N intervals a step while every
|
||||
shared step is planned at a smaller shared budget, which on an expert-tiered box lends that arena
|
||||
back to the expert slab. The vendor's own measurement says the win is **long-prompt-only**: nothing
|
||||
at short prompts, a large TTFT gain at the far end of the window.
|
||||
|
||||
Two traps. **The arena/slab coupling makes a shared step doubly expensive on an
|
||||
expert-tiered shape**: the full-plan arena takes VRAM the resident expert slab would
|
||||
have had (320 MiB/card at TP2 for a 4096 budget), so the decode rows inside that step
|
||||
also read experts over the link — a smaller shared budget helps twice. And **re-check
|
||||
expert-tiered shape**: the full-plan arena takes VRAM the resident expert slab would have had, so the
|
||||
decode rows inside that step also read experts over the link — a smaller shared budget helps twice.
|
||||
And **re-check
|
||||
`prefix_hit_rate` / `prefix_evictions` after any chunk change**: the arena and the KV
|
||||
pool are the same memory, and the failure looks like turn two taking as long as turn
|
||||
one.
|
||||
@@ -276,8 +275,9 @@ Availability is a version question, not a config question: `--busy-prefill-chunk
|
||||
tree, and check the source clone's depth before dating a flag from `git log -S`
|
||||
(a `--depth 1` clone reports its own tip for every search).
|
||||
|
||||
Run on: [[frank-archon]] (radiance 1.0.13, TP2 `expert_tiered`, budget 2048 = interval
|
||||
2048) — the measured census and the verdict for this fleet's shape are that page's.
|
||||
Run on: [[frank-archon]] — the census was taken on its 3-slot radiance 1.0.13 era (budget 2048 =
|
||||
interval 2048); the live 1.8.1 shape carries the two chunk knobs. The measured census and the verdict
|
||||
for this fleet's shape are that page's.
|
||||
|
||||
## Related
|
||||
|
||||
|
||||
Reference in New Issue
Block a user