frank-large: record deployed boot facts (pool 819,332 tokens, 3.13x, 230 s)
Live battery green after deploy #3: alias, reasoning split into `reasoning`, tool_calls, /metrics acceptance pos0 69% pos1 44% - the cross-model MTP head performs as measured in the bench window.
This commit is contained in:
@@ -72,6 +72,9 @@ FRANK_LARGE_PUBLISH=7770:7770
|
||||
FRANK_LARGE_MAXLEN=262144
|
||||
FRANK_LARGE_SEQS=4
|
||||
FRANK_LARGE_UTIL=0.45
|
||||
# First-deploy boot (2026-10-01): GPU KV cache size 819,332 tokens,
|
||||
# Maximum concurrency for 262,144 tokens per request 3.13x; healthy at 230 s.
|
||||
# Change MAXLEN/SEQS/UTIL or the image and this line moves — re-read it.
|
||||
|
||||
# --- speculative decode: cross-model MTP (measured +51% decode) ---------------
|
||||
# mtp beats dflash here (19.8 vs 17.4 t/s) and needs 2 draft tokens, not 8:
|
||||
|
||||
@@ -155,8 +155,13 @@ chase text drift between spec and no-spec arms as a regression.
|
||||
frank-primary lesson).
|
||||
- Bench windows: `systemctl stop frank-large` is enough (mask fails on the
|
||||
relinked unit; the deploy restores).
|
||||
- First-deploy fact to record here: the boot-time KV pool line
|
||||
(`Maximum concurrency for 262,144 tokens per prompt: N.Nx`).
|
||||
- First-deploy facts (2026-10-01, deploy #3): healthy after **230 s**;
|
||||
boot log `GPU KV cache size: 819,332 tokens, Maximum concurrency for
|
||||
262,144 tokens per request: 3.13x` — four sessions share the pool
|
||||
(the scheduler, not a carve). Live battery green: alias `frank-large`,
|
||||
reasoning split into `reasoning`, tool_calls with correct args, and
|
||||
`/metrics` acceptance pos0 69% pos1 44% (54/86 draft tokens accepted),
|
||||
matching the bench window.
|
||||
|
||||
## Rollback
|
||||
|
||||
|
||||
Reference in New Issue
Block a user