docs: frank-secondary at ubatch 2048 — Measured section, rollback ladder, fleet docs
23,120 used / 1,005 free at boot, 763 settled after a vision probe (encoder buffers resident). Rung cost 746 MiB, not the transferable 616 prediction. Prefill baseline 1,209 t/s @ 3,760 recorded; decode unchanged as the law says. 2048 = ceiling for this shape.
This commit is contained in:
@@ -2,6 +2,23 @@
|
||||
|
||||
|
||||
## 2026-10-06
|
||||
- **frank-secondary (Dankclaw): ubatch 2048 — the second rung (llama.cpp,
|
||||
RTX 3090).** "set ubatch to 2k" → commit d8da7a9: `UBATCH_SIZE`
|
||||
1024 → 2048 at the 2×114,688 shape. **Cost 746 MiB**, not the 616 MiB
|
||||
the same rung measured on the MiMo shape on 2026-10-05 (a rung's price
|
||||
is shape-dependent; the 616 figure was not transferable). Boot:
|
||||
**23,120 used / 1,005 MiB free**; a 512×512 vision probe then took
|
||||
~240 MiB **and kept it** — settled **763 MiB free**, the encoder's
|
||||
buffers staying resident across requests on this build, unlike the
|
||||
evening boot where the 34 MiB transient returned. Decode 52.0 t/s on
|
||||
the shared prompt (58.2 at the 1024 rung) with acceptance 34.2 % vs
|
||||
42.8 %: ubatch does not move decode, so the gap is temp-1.0
|
||||
acceptance noise, not a regression. **Prefill 1,209 t/s @ 3,760
|
||||
tokens** — the first prefill number this endpoint has recorded, and
|
||||
now the baseline future rungs compare against. `BATCH_SIZE` stays
|
||||
4096, so the seam's batch ≥ ubatch guard holds; **2048 is the ceiling
|
||||
for this shape** (4096 = no micro-split doubles the buffer growth
|
||||
again against 0.7–1.0 GiB of headroom). e2e gate 33/33 green.
|
||||
- **frank-secondary (Dankclaw): RENAMED from frank-small, then reshaped
|
||||
to 2×114,688 / ubatch 1024 (llama.cpp, RTX 3090).** The rename (commit
|
||||
87721ab, 87 files: serve trio, unit, bats seam, 9 runbooks, identifiers
|
||||
|
||||
Reference in New Issue
Block a user