docs: frank-secondary at ubatch 2048 — Measured section, rollback ladder, fleet docs
23,120 used / 1,005 free at boot, 763 settled after a vision probe (encoder buffers resident). Rung cost 746 MiB, not the transferable 616 prediction. Prefill baseline 1,209 t/s @ 3,760 recorded; decode unchanged as the law says. 2048 = ceiling for this shape.
This commit is contained in:
1 parent
d8da7a9eb7
commit
41d2f56269
5 files changed
+104
-51
No files matched your search
@@ -376,11 +376,11 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
frank-secondary runbook since 2026-10-05 (evening; **UNCECERSED RCO-IQ3_XXS
|
||||
with MTP back on since 2026-10-06 evening; renamed from frank-small and
|
||||
reshaped to TWO 114,688-token conversations at Frank's 112k cap with
|
||||
ubatch 1024 late 2026-10-06**): llama.cpp master (CUDA
|
||||
the ubatch ladder climbed to 2048 late 2026-10-06**): llama.cpp master (CUDA
|
||||
sm_86) serving ajgazin's Swift-1.5-Qwen3.8-27B Uncensored RCO-IQ3_XXS
|
||||
(the MTP head in the file is drafted) on the RTX 3090 — 1,751 MiB
|
||||
measured free, q8_0 K + q8_0 V, vision
|
||||
through the publisher's own projector, ~58 t/s recipe decode with MTP
|
||||
(the MTP head in the file is drafted) on the RTX 3090 — 1,005 MiB free
|
||||
at boot (763 settled after a vision probe), q8_0 K + q8_0 V, vision
|
||||
through the publisher's own projector, ~52–58 t/s recipe decode with MTP
|
||||
- [[frank-secondary-llama-mimo-q6k-2x200704]] — RETIRED 2026-10-05 (evening,
|
||||
superseded by swift15): llama.cpp master serving
|
||||
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** — TWO conversations at
|
||||
|
||||
Reference in new issue
Block a user