frank-small: record measured cutover values; unit Description follows the Swift swap
Boot ledger lines, 19,404 used / 4,721 MiB free (the q8_0 proof), decode arms (66.1 t/s @0.8, 66.9 @0, 58.9 @recipe 1.0 with acceptance 52.1%/52.2%/44.8%), vision Red probe, reasoning_effort xhigh/high contract, e2e 12-crawl transient + full-gate green rerun. Unit Description updated in-repo; the relinked unit picks it up at the next deploy/restart (no bounce for cosmetics). 360/360 suite green.
This commit is contained in:
1 parent
baf33d016f
commit
13112a93f5
4 files changed
+51
-9
No files matched your search
@@ -408,12 +408,20 @@ conversation at Frank's 128k cap (Swift's own benchmark protocol ran at
|
||||
class — no true fp8 KV dtype exists; 8.5 effective bits; the order
|
||||
SUPERSEDES the same-day f16/f16 rule written for the Ornith 9B fit), all
|
||||
layers on the GPU, `--mmproj` for vision, batch 4096 / ubatch 1024 (lower
|
||||
ubatch first, Frank's standing law; 2048 is the first A/B if boot lands
|
||||
high). Non-KV ≈ 15.5 GiB (weights 11,561 + projector ~1,350 + buffers
|
||||
~1.6k + CUDA ~500 + DeltaNet ~150/slot + MTP draft ctx ~0.3-0.5k); one q8_0
|
||||
slot = 4,352 MiB ⇒ free ≈ 4.5 GiB; a SECOND slot wants +4,352 ⇒ ~0.2 GiB —
|
||||
far under the 2-3 GiB floor. The pool is capped by the 128k law, not the
|
||||
card; leftover headroom is honest slack.
|
||||
ubatch first, Frank's standing law; 2048 is the first A/B if more headroom
|
||||
is ever wanted). Measured at boot: **19,404 used / 4,721 MiB free**
|
||||
(`nvidia-smi` free field — the 4.5 GiB prediction held; f16 KV would have
|
||||
left ~700 MiB, so the free number is also the q8_0 proof). The pool is
|
||||
capped by the 128k law, not the card; a SECOND slot wants +4,352 MiB and
|
||||
would land ~0.2 GiB free — far under the 2-3 GiB floor.
|
||||
|
||||
**Measured decode (2026-10-05 probes, draft_n counts from the live
|
||||
`timings`)**: temp 0.8 → **66.1 tok/s**, acceptance 182/349 (52.1 %); temp 0
|
||||
→ 66.9 tok/s, 166/318; **deployed recipe temp 1.0/top_p 0.95/top_k 20 →
|
||||
58.9 tok/s, 159/355 (44.8 %)** — the higher temperature costs both acceptance
|
||||
and rate. Vision: a 512×512 solid-red PNG answered `Red` through an
|
||||
`image_url` part. `reasoning_effort` probe: `xhigh` 200, `high` → 500
|
||||
Jinja raise (the models.yml fold stays mandatory).
|
||||
|
||||
**Speculative decoding is the file's own MTP head**: the `-mtp` GGUF
|
||||
carries `blk.64.nextn.*` (`qwen35.nextn_predict_layers=1`), so
|
||||
|
||||
Reference in new issue
Block a user