docs(frank-small): record the 3 x 180,224 + vision shape and what it cost

Runbook, both indexes, AGENTS.md (topology row, endpoint section, three new
Key Lessons, one stale SGLang label in the bench list) and the bundle log now
describe the deployed shape: pool 540,672, 180,224 tokens per conversation,
f16 K + q8_0 V, mmproj vision, 21,684 used / 2,441 free of 24,576 MiB, 106.8
tok/s decode, 3,814 t/s prefill.

The claims that mattered:
- three contexts buy 180,224 tokens per conversation, not the native 262,144,
  and a longer prompt is a hard 400 - so models.yml contextWindow is 180224
  and frank-primary's 183,501-token compaction request no longer fits here
  (forward pointer added to the retired SGLang runbook, which asserted it)
- the vision tower bills ~1.4 GiB for a 918 MB file, established by subtracting
  two measured boots, not by a formula
- /v1/chat/completions ignores the legacy top-level 'images' field: the model
  invents a color; only the image_url content part reaches it
This commit is contained in:
Frank committed 2026-10-02 03:46:50 -07:00
1 parent 7fe881de01
commit 934285c2b2
7 files changed
+245 -67

No files matched your search

@@ -52,7 +52,10 @@ timestamp: 2026-10-01
trigger 183,501 + a 32,768-token summary budget = 216,269) now fits
one request. A single request may grow to the whole pool — a
170,019-token prompt was verified at the 4-slot shape — the grid is
the two-way concurrency contract.
the two-way concurrency contract. **That compaction arithmetic is
specific to a 262,144-token slot**: the ACTIVE shape runs three slots of
180,224, which 183,501 tokens already exceeds — see *What three slots
cost* in [[frank-small-llama-mimo-q5km]].
- **KV:** `fp8_e4m3` — 16 KiB/token (8 attn layers x 2 x 4 heads x 256
dim x 1 B), pool 9.31 GiB.
- **The memory shape (the 4.5 GiB the naive math misses):** the DeltaNet