diff --git a/serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md b/serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md index ef348b5..648daf3 100644 --- a/serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md +++ b/serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md @@ -142,9 +142,15 @@ re-checks). The serve script refuses to start with a missing shard. compiles the sgl-kernel + flashinfer kernels; the CUDA 13 toolchain above is what makes that compile succeed). - Decode **83.9 tok/s** (no spec decode — nothing to accept). -- Prefill ~2,950–3,370 t/s eager (a 170,019-token prompt in 50.5 s). -- 4 concurrent 54,027-token requests all answer in ~17 s (216k tokens of - live KV at once). +- Prefill ~2,050–3,370 t/s eager (4-slot shape: a 170,019-token prompt in + 50.5 s, ~3,370 t/s; 2-slot shape: a 200,000-token prompt in 97.3 s wall + incl. 8 decode tokens, ~2,056 t/s). +- 4-slot shape: 4 concurrent 54,027-token requests all answered in ~17 s + (216k tokens of live KV at once). 2-slot shape: two concurrent + 50,000-token requests both answered (28.8 s wall). +- **Compaction fit (the reslice's purpose):** a 200,000-token prompt — + well past frank-primary's 183,501-token compaction trigger (70% of its + 262,144 window) — prefills and completes in ONE request. - Vision: a 64x64 red square answers "Red" with `image_tokens: 64`. ## Rollback