From db54fa3af1d97661066fc22695d187547b773d2f Mon Sep 17 00:00:00 2001 From: Frank Date: Thu, 1 Oct 2026 14:08:51 -0700 Subject: [PATCH] frank-small: record 2x262144 verification numbers Live-verified post-deploy: context_len=262144, max_running_requests=2, max_total_num_tokens=606208 in the boot log; a 200,000-token prompt (compaction height is 183,501) prefilled+completed in one request (97.3 s, ~2,056 tok/s eager); two concurrent 50k requests answered. Docs-only change. --- .../LLM/frank-small-sglang-mimo-v26.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md b/serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md index ef348b5..648daf3 100644 --- a/serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md +++ b/serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md @@ -142,9 +142,15 @@ re-checks). The serve script refuses to start with a missing shard. compiles the sgl-kernel + flashinfer kernels; the CUDA 13 toolchain above is what makes that compile succeed). - Decode **83.9 tok/s** (no spec decode — nothing to accept). -- Prefill ~2,950–3,370 t/s eager (a 170,019-token prompt in 50.5 s). -- 4 concurrent 54,027-token requests all answer in ~17 s (216k tokens of - live KV at once). +- Prefill ~2,050–3,370 t/s eager (4-slot shape: a 170,019-token prompt in + 50.5 s, ~3,370 t/s; 2-slot shape: a 200,000-token prompt in 97.3 s wall + incl. 8 decode tokens, ~2,056 t/s). +- 4-slot shape: 4 concurrent 54,027-token requests all answered in ~17 s + (216k tokens of live KV at once). 2-slot shape: two concurrent + 50,000-token requests both answered (28.8 s wall). +- **Compaction fit (the reslice's purpose):** a 200,000-token prompt — + well past frank-primary's 183,501-token compaction trigger (70% of its + 262,144 window) — prefills and completes in ONE request. - Vision: a 64x64 red square answers "Red" with `image_tokens: 64`. ## Rollback