docs: frank-secondary at ubatch 2048 — Measured section, rollback ladder, fleet docs

23,120 used / 1,005 free at boot, 763 settled after a vision probe
(encoder buffers resident). Rung cost 746 MiB, not the transferable
616 prediction. Prefill baseline 1,209 t/s @ 3,760 recorded; decode
unchanged as the law says. 2048 = ceiling for this shape.
This commit is contained in:
Frank
2026-10-06 18:40:26 -07:00
parent d8da7a9eb7
commit 41d2f56269
5 changed files with 104 additions and 51 deletions
+17
View File
@@ -2,6 +2,23 @@
## 2026-10-06
- **frank-secondary (Dankclaw): ubatch 2048 — the second rung (llama.cpp,
RTX 3090).** "set ubatch to 2k" → commit d8da7a9: `UBATCH_SIZE`
1024 → 2048 at the 2×114,688 shape. **Cost 746 MiB**, not the 616 MiB
the same rung measured on the MiMo shape on 2026-10-05 (a rung's price
is shape-dependent; the 616 figure was not transferable). Boot:
**23,120 used / 1,005 MiB free**; a 512×512 vision probe then took
~240 MiB **and kept it** — settled **763 MiB free**, the encoder's
buffers staying resident across requests on this build, unlike the
evening boot where the 34 MiB transient returned. Decode 52.0 t/s on
the shared prompt (58.2 at the 1024 rung) with acceptance 34.2 % vs
42.8 %: ubatch does not move decode, so the gap is temp-1.0
acceptance noise, not a regression. **Prefill 1,209 t/s @ 3,760
tokens** — the first prefill number this endpoint has recorded, and
now the baseline future rungs compare against. `BATCH_SIZE` stays
4096, so the seam's batch ≥ ubatch guard holds; **2048 is the ceiling
for this shape** (4096 = no micro-split doubles the buffer growth
again against 0.7–1.0 GiB of headroom). e2e gate 33/33 green.
- **frank-secondary (Dankclaw): RENAMED from frank-small, then reshaped
to 2×114,688 / ubatch 1024 (llama.cpp, RTX 3090).** The rename (commit
87721ab, 87 files: serve trio, unit, bats seam, 9 runbooks, identifiers