frank-small: record measured cutover values; unit Description follows the Swift swap

Boot ledger lines, 19,404 used / 4,721 MiB free (the q8_0 proof), decode arms (66.1 t/s @0.8, 66.9 @0, 58.9 @recipe 1.0 with acceptance 52.1%/52.2%/44.8%), vision Red probe, reasoning_effort xhigh/high contract, e2e 12-crawl transient + full-gate green rerun. Unit Description updated in-repo; the relinked unit picks it up at the next deploy/restart (no bounce for cosmetics). 360/360 suite green.
This commit is contained in:
Frank committed 2026-10-05 23:41:05 -07:00
1 parent baf33d016f
commit 13112a93f5
4 files changed
+51 -9

No files matched your search

+14 -6
View File
@@ -408,12 +408,20 @@ conversation at Frank's 128k cap (Swift's own benchmark protocol ran at
class — no true fp8 KV dtype exists; 8.5 effective bits; the order
SUPERSEDES the same-day f16/f16 rule written for the Ornith 9B fit), all
layers on the GPU, `--mmproj` for vision, batch 4096 / ubatch 1024 (lower
ubatch first, Frank's standing law; 2048 is the first A/B if boot lands
high). Non-KV ≈ 15.5 GiB (weights 11,561 + projector ~1,350 + buffers
~1.6k + CUDA ~500 + DeltaNet ~150/slot + MTP draft ctx ~0.3-0.5k); one q8_0
slot = 4,352 MiB ⇒ free ≈ 4.5 GiB; a SECOND slot wants +4,352 ⇒ ~0.2 GiB —
far under the 2-3 GiB floor. The pool is capped by the 128k law, not the
card; leftover headroom is honest slack.
ubatch first, Frank's standing law; 2048 is the first A/B if more headroom
is ever wanted). Measured at boot: **19,404 used / 4,721 MiB free**
(`nvidia-smi` free field — the 4.5 GiB prediction held; f16 KV would have
left ~700 MiB, so the free number is also the q8_0 proof). The pool is
capped by the 128k law, not the card; a SECOND slot wants +4,352 MiB and
would land ~0.2 GiB free — far under the 2-3 GiB floor.
**Measured decode (2026-10-05 probes, draft_n counts from the live
`timings`)**: temp 0.8 → **66.1 tok/s**, acceptance 182/349 (52.1 %); temp 0
→ 66.9 tok/s, 166/318; **deployed recipe temp 1.0/top_p 0.95/top_k 20 →
58.9 tok/s, 159/355 (44.8 %)** — the higher temperature costs both acceptance
and rate. Vision: a 512×512 solid-red PNG answered `Red` through an
`image_url` part. `reasoning_effort` probe: `xhigh` 200, `high` → 500
Jinja raise (the models.yml fold stays mandatory).
**Speculative decoding is the file's own MTP head**: the `-mtp` GGUF
carries `blk.64.nextn.*` (`qwen35.nextn_predict_layers=1`), so