frank-small: record 2x262144 verification numbers

Live-verified post-deploy: context_len=262144, max_running_requests=2,
max_total_num_tokens=606208 in the boot log; a 200,000-token prompt
(compaction height is 183,501) prefilled+completed in one request
(97.3 s, ~2,056 tok/s eager); two concurrent 50k requests answered.
Docs-only change.
This commit is contained in:
Frank
2026-10-01 14:08:51 -07:00
parent 8029119b19
commit db54fa3af1
@@ -142,9 +142,15 @@ re-checks). The serve script refuses to start with a missing shard.
compiles the sgl-kernel + flashinfer kernels; the CUDA 13 toolchain
above is what makes that compile succeed).
- Decode **83.9 tok/s** (no spec decode — nothing to accept).
- Prefill ~2,950–3,370 t/s eager (a 170,019-token prompt in 50.5 s).
- 4 concurrent 54,027-token requests all answer in ~17 s (216k tokens of
live KV at once).
- Prefill ~2,050–3,370 t/s eager (4-slot shape: a 170,019-token prompt in
50.5 s, ~3,370 t/s; 2-slot shape: a 200,000-token prompt in 97.3 s wall
incl. 8 decode tokens, ~2,056 t/s).
- 4-slot shape: 4 concurrent 54,027-token requests all answered in ~17 s
(216k tokens of live KV at once). 2-slot shape: two concurrent
50,000-token requests both answered (28.8 s wall).
- **Compaction fit (the reslice's purpose):** a 200,000-token prompt —
well past frank-primary's 183,501-token compaction trigger (70% of its
262,144 window) — prefills and completes in ONE request.
- Vision: a 64x64 red square answers "Red" with `image_tokens: 64`.
## Rollback