frank-large: record deployed boot facts (pool 819,332 tokens, 3.13x, 230 s)

Live battery green after deploy #3: alias, reasoning split into
`reasoning`, tool_calls, /metrics acceptance pos0 69% pos1 44% - the
cross-model MTP head performs as measured in the bench window.
This commit is contained in:
Frank
2026-10-01 19:48:56 -07:00
parent 780c307354
commit 3c339eeba2
2 changed files with 10 additions and 2 deletions
+3
View File
@@ -72,6 +72,9 @@ FRANK_LARGE_PUBLISH=7770:7770
FRANK_LARGE_MAXLEN=262144
FRANK_LARGE_SEQS=4
FRANK_LARGE_UTIL=0.45
# First-deploy boot (2026-10-01): GPU KV cache size 819,332 tokens,
# Maximum concurrency for 262,144 tokens per request 3.13x; healthy at 230 s.
# Change MAXLEN/SEQS/UTIL or the image and this line moves — re-read it.
# --- speculative decode: cross-model MTP (measured +51% decode) ---------------
# mtp beats dflash here (19.8 vs 17.4 t/s) and needs 2 draft tokens, not 8:
@@ -155,8 +155,13 @@ chase text drift between spec and no-spec arms as a regression.
frank-primary lesson).
- Bench windows: `systemctl stop frank-large` is enough (mask fails on the
relinked unit; the deploy restores).
- First-deploy fact to record here: the boot-time KV pool line
(`Maximum concurrency for 262,144 tokens per prompt: N.Nx`).
- First-deploy facts (2026-10-01, deploy #3): healthy after **230 s**;
boot log `GPU KV cache size: 819,332 tokens, Maximum concurrency for
262,144 tokens per request: 3.13x` — four sessions share the pool
(the scheduler, not a carve). Live battery green: alias `frank-large`,
reasoning split into `reasoning`, tool_calls with correct args, and
`/metrics` acceptance pos0 69% pos1 44% (54/86 draft tokens accepted),
matching the bench window.
## Rollback