frank-small docs: runbook renamed + rewritten for the RCO-IQ3_XXS MTP-on shape (489 MiB free, 63.3 t/s recipe, draft ctx ~2.4 GiB measured); AGENTS, vault indexes, models.yml comment, MiMo cross-link retargeted

This commit is contained in:
Frank committed 2026-10-06 01:33:57 -07:00
1 parent da36fa32cf
commit 0f7274e55a
6 files changed
+360 -316

No files matched your search

+25 -24
View File
@@ -15,11 +15,12 @@ ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
/halogen-qwen3.8-27b, a 33.4 GiB .hgn checkpoint that loads in no other
engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
lives on frank-small)** and
**llama.cpp CUDA (frank-small, port 7771: ukisai Swift-1.5-Qwen3.8-27B
GSQ-RCO IQ3_S-mtp GGUF — 27B (head present, NOT drafted), TWO 131,072
conversations at Frank's 128k cap (tight fit, spec off since 2026-10-06),
q8_0 K + q8_0 V (FP8 class), vision via the base-model projector, on the
RTX 3090)**.
**llama.cpp CUDA (frank-small, port 7771: ajgazin Swift-1.5-Qwen3.8-27B
Uncensored Dynamic-MTP RCO-IQ3_XXS GGUF — 27B (the MTP head in the file
IS drafted, draft-mtp depth 3), TWO 131,072 conversations at Frank's
128k cap (the tightest fit in the endpoint's history — 489 MiB free),
q8_0 K + q8_0 V (FP8 class), vision via the publisher's own BF16
mmproj, on the RTX 3090)**.
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -50,8 +51,8 @@ frank-large, port 7770]
1 x 200700 ctx, d2 KV, text-only]
G[llama.cpp master CUDA sm_86
frank-small, port 7771]
H[Swift-1.5-27B GSQ-RCO IQ3_S-mtp GGUF
1 x 131072 ctx, q8_0 K + q8_0 V, draft-mtp depth 3, mmproj vision]
H[Swift-1.5-27B Uncensored RCO-IQ3_XXS-mtp GGUF
2 x 131072 ctx, q8_0 K + q8_0 V, draft-mtp depth 3, mmproj vision]
E2 --> E --> F
E2 --> G --> H
end
@@ -94,23 +95,23 @@ frank-small, port 7771]
on this topology — the default tile stands. The deep harness roles
(`slow`, `advisor`) rerouted to frank-primary; frank-large keeps
OpenWebUI and shallow callers.
- [[frank-small-llama-swift15-iq3s-mtp-131072]] — **ACTIVE since 2026-10-05
(evening); 2 × 131,072 no-spec since 2026-10-06** (root systemd unit,
port 7771): llama.cpp master (CUDA sm_86) serving ukisai/Swift-1.5-
Qwen3.8-27B-GSQ-RCO-GGUF **IQ3_S-mtp** (27B; the -mtp file's head is
present but NOT drafted — speculation OFF by Frank's order, ngram skipped
on the recurrent-rollback law) on the RTX 3090, TWO 131,072-token
conversations at Frank's 128k cap (pool 262144 / parallel 2 — the tight
fit he ordered: **1,397 MiB measured free**, the 2-3 GiB floor knowingly
violated) with q8_0 K + q8_0 V (FP8 class, 34 KiB/token vs 64 at f16),
ubatch 512, and vision via unsloth's mmproj-Qwen3.8-27B-F16.gguf — no
Swift projector ships, and the base vision stack was hash-proven
byte-identical to Swift's (6/6 tensors). Decode 48.0 t/s without the
drafter (MTP measured 58.9–66.9; references kept in frank-small.env).
The GSQ-RCO "IQ3_S" tier is a per-tensor allocation mix of stock types.
Replaces BOTH same-day payloads: the live MiMo Q6_K shape (below) and
the Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but
never deployed
- [[frank-small-llama-swift15-unc-iq3xxs-mtp-131072]] — **ACTIVE since
2026-10-05 (evening); UNCECERSED RCO-IQ3_XXS with MTP back on since
2026-10-06 evening** (root systemd unit, port 7771): llama.cpp master
(CUDA sm_86) serving ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-
GGUF **RCO-IQ3_XXS-mtp** (27B; the MTP head in the file is DRAFTED —
draft-mtp depth 3, ngram skipped on the recurrent-rollback law) on the
RTX 3090, TWO 131,072-token conversations at Frank's 128k cap (pool
262144 / parallel 2 — the tightest fit in the endpoint's history:
**489 MiB measured free**, the 2-3 GiB floor knowingly violated) with
q8_0 K + q8_0 V (FP8 class, 34 KiB/token vs 64 at f16), ubatch 512, and
vision through the publisher's own mmproj-BF16.gguf. Decode 63.3 t/s
recipe / 72.9 temp 0 (MTP acceptance 49.8–62.4 %; 48.0 no-spec). The
RCO-IQ3_XXS tier is a per-tensor allocation mix of stock types; the
payload is the UNCECERSED Swift 1.5 (23/100 refusals vs Swift 1.5's
98/100). Replaces BOTH same-day payloads: the live MiMo Q6_K shape
(below) and the Ornith-1.5-9B Q8_0 swap that was committed
(77decb5..380aafa) but never deployed
- [[frank-small-llama-mimo-q6k-2x200704]] — **RETIRED 2026-10-05 (evening;
rollback target; payload on disk)**: llama.cpp master (CUDA sm_86)
serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K**, TWO slots of