docs: frank-secondary at ubatch 2048 — Measured section, rollback ladder, fleet docs

23,120 used / 1,005 free at boot, 763 settled after a vision probe
(encoder buffers resident). Rung cost 746 MiB, not the transferable
616 prediction. Prefill baseline 1,209 t/s @ 3,760 recorded; decode
unchanged as the law says. 2048 = ceiling for this shape.
This commit is contained in:
Frank committed 2026-10-06 18:40:26 -07:00
1 parent d8da7a9eb7
commit 41d2f56269
5 files changed
+104 -51

No files matched your search

+4 -4
View File
@@ -376,11 +376,11 @@ The LLM inference fleet on the two hosts. Full index:
frank-secondary runbook since 2026-10-05 (evening; **UNCECERSED RCO-IQ3_XXS
with MTP back on since 2026-10-06 evening; renamed from frank-small and
reshaped to TWO 114,688-token conversations at Frank's 112k cap with
ubatch 1024 late 2026-10-06**): llama.cpp master (CUDA
the ubatch ladder climbed to 2048 late 2026-10-06**): llama.cpp master (CUDA
sm_86) serving ajgazin's Swift-1.5-Qwen3.8-27B Uncensored RCO-IQ3_XXS
(the MTP head in the file is drafted) on the RTX 3090 — 1,751 MiB
measured free, q8_0 K + q8_0 V, vision
through the publisher's own projector, ~58 t/s recipe decode with MTP
(the MTP head in the file is drafted) on the RTX 3090 — 1,005 MiB free
at boot (763 settled after a vision probe), q8_0 K + q8_0 V, vision
through the publisher's own projector, ~52–58 t/s recipe decode with MTP
- [[frank-secondary-llama-mimo-q6k-2x200704]] — RETIRED 2026-10-05 (evening,
superseded by swift15): llama.cpp master serving
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** — TWO conversations at