frank-large: record deployed boot facts (pool 819,332 tokens, 3.13x, 230 s)
Live battery green after deploy #3: alias, reasoning split into `reasoning`, tool_calls, /metrics acceptance pos0 69% pos1 44% - the cross-model MTP head performs as measured in the bench window.
This commit is contained in:
1 parent
780c307354
commit
3c339eeba2
2 files changed
+10
-2
No files matched your search
@@ -72,6 +72,9 @@ FRANK_LARGE_PUBLISH=7770:7770
|
|||||||
FRANK_LARGE_MAXLEN=262144
|
FRANK_LARGE_MAXLEN=262144
|
||||||
FRANK_LARGE_SEQS=4
|
FRANK_LARGE_SEQS=4
|
||||||
FRANK_LARGE_UTIL=0.45
|
FRANK_LARGE_UTIL=0.45
|
||||||
|
# First-deploy boot (2026-10-01): GPU KV cache size 819,332 tokens,
|
||||||
|
# Maximum concurrency for 262,144 tokens per request 3.13x; healthy at 230 s.
|
||||||
|
# Change MAXLEN/SEQS/UTIL or the image and this line moves — re-read it.
|
||||||
|
|
||||||
# --- speculative decode: cross-model MTP (measured +51% decode) ---------------
|
# --- speculative decode: cross-model MTP (measured +51% decode) ---------------
|
||||||
# mtp beats dflash here (19.8 vs 17.4 t/s) and needs 2 draft tokens, not 8:
|
# mtp beats dflash here (19.8 vs 17.4 t/s) and needs 2 draft tokens, not 8:
|
||||||
|
|||||||
@@ -155,8 +155,13 @@ chase text drift between spec and no-spec arms as a regression.
|
|||||||
frank-primary lesson).
|
frank-primary lesson).
|
||||||
- Bench windows: `systemctl stop frank-large` is enough (mask fails on the
|
- Bench windows: `systemctl stop frank-large` is enough (mask fails on the
|
||||||
relinked unit; the deploy restores).
|
relinked unit; the deploy restores).
|
||||||
- First-deploy fact to record here: the boot-time KV pool line
|
- First-deploy facts (2026-10-01, deploy #3): healthy after **230 s**;
|
||||||
(`Maximum concurrency for 262,144 tokens per prompt: N.Nx`).
|
boot log `GPU KV cache size: 819,332 tokens, Maximum concurrency for
|
||||||
|
262,144 tokens per request: 3.13x` — four sessions share the pool
|
||||||
|
(the scheduler, not a carve). Live battery green: alias `frank-large`,
|
||||||
|
reasoning split into `reasoning`, tool_calls with correct args, and
|
||||||
|
`/metrics` acceptance pos0 69% pos1 44% (54/86 draft tokens accepted),
|
||||||
|
matching the bench window.
|
||||||
|
|
||||||
## Rollback
|
## Rollback
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user