diff --git a/scripts/frank-large.env b/scripts/frank-large.env index 05dab9e..256f893 100644 --- a/scripts/frank-large.env +++ b/scripts/frank-large.env @@ -72,6 +72,9 @@ FRANK_LARGE_PUBLISH=7770:7770 FRANK_LARGE_MAXLEN=262144 FRANK_LARGE_SEQS=4 FRANK_LARGE_UTIL=0.45 +# First-deploy boot (2026-10-01): GPU KV cache size 819,332 tokens, +# Maximum concurrency for 262,144 tokens per request 3.13x; healthy at 230 s. +# Change MAXLEN/SEQS/UTIL or the image and this line moves — re-read it. # --- speculative decode: cross-model MTP (measured +51% decode) --------------- # mtp beats dflash here (19.8 vs 17.4 t/s) and needs 2 draft tokens, not 8: diff --git a/serverAppConfigs-docs/LLM/frank-large-vllm-mimo-v26-bf16.md b/serverAppConfigs-docs/LLM/frank-large-vllm-mimo-v26-bf16.md index cef2dec..fa1b71b 100644 --- a/serverAppConfigs-docs/LLM/frank-large-vllm-mimo-v26-bf16.md +++ b/serverAppConfigs-docs/LLM/frank-large-vllm-mimo-v26-bf16.md @@ -155,8 +155,13 @@ chase text drift between spec and no-spec arms as a regression. frank-primary lesson). - Bench windows: `systemctl stop frank-large` is enough (mask fails on the relinked unit; the deploy restores). -- First-deploy fact to record here: the boot-time KV pool line - (`Maximum concurrency for 262,144 tokens per prompt: N.Nx`). +- First-deploy facts (2026-10-01, deploy #3): healthy after **230 s**; + boot log `GPU KV cache size: 819,332 tokens, Maximum concurrency for + 262,144 tokens per request: 3.13x` — four sessions share the pool + (the scheduler, not a carve). Live battery green: alias `frank-large`, + reasoning split into `reasoning`, tool_calls with correct args, and + `/metrics` acceptance pos0 69% pos1 44% (54/86 draft tokens accepted), + matching the bench window. ## Rollback