frank-small docs: runbook renamed + rewritten for the RCO-IQ3_XXS MTP-on shape (489 MiB free, 63.3 t/s recipe, draft ctx ~2.4 GiB measured); AGENTS, vault indexes, models.yml comment, MiMo cross-link retargeted
This commit is contained in:
1 parent
da36fa32cf
commit
0f7274e55a
6 files changed
+360
-316
No files matched your search
@@ -15,11 +15,12 @@ ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
|
||||
/halogen-qwen3.8-27b, a 33.4 GiB .hgn checkpoint that loads in no other
|
||||
engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
|
||||
lives on frank-small)** and
|
||||
**llama.cpp CUDA (frank-small, port 7771: ukisai Swift-1.5-Qwen3.8-27B
|
||||
GSQ-RCO IQ3_S-mtp GGUF — 27B (head present, NOT drafted), TWO 131,072
|
||||
conversations at Frank's 128k cap (tight fit, spec off since 2026-10-06),
|
||||
q8_0 K + q8_0 V (FP8 class), vision via the base-model projector, on the
|
||||
RTX 3090)**.
|
||||
**llama.cpp CUDA (frank-small, port 7771: ajgazin Swift-1.5-Qwen3.8-27B
|
||||
Uncensored Dynamic-MTP RCO-IQ3_XXS GGUF — 27B (the MTP head in the file
|
||||
IS drafted, draft-mtp depth 3), TWO 131,072 conversations at Frank's
|
||||
128k cap (the tightest fit in the endpoint's history — 489 MiB free),
|
||||
q8_0 K + q8_0 V (FP8 class), vision via the publisher's own BF16
|
||||
mmproj, on the RTX 3090)**.
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -50,8 +51,8 @@ frank-large, port 7770]
|
||||
1 x 200700 ctx, d2 KV, text-only]
|
||||
G[llama.cpp master CUDA sm_86
|
||||
frank-small, port 7771]
|
||||
H[Swift-1.5-27B GSQ-RCO IQ3_S-mtp GGUF
|
||||
1 x 131072 ctx, q8_0 K + q8_0 V, draft-mtp depth 3, mmproj vision]
|
||||
H[Swift-1.5-27B Uncensored RCO-IQ3_XXS-mtp GGUF
|
||||
2 x 131072 ctx, q8_0 K + q8_0 V, draft-mtp depth 3, mmproj vision]
|
||||
E2 --> E --> F
|
||||
E2 --> G --> H
|
||||
end
|
||||
@@ -94,23 +95,23 @@ frank-small, port 7771]
|
||||
on this topology — the default tile stands. The deep harness roles
|
||||
(`slow`, `advisor`) rerouted to frank-primary; frank-large keeps
|
||||
OpenWebUI and shallow callers.
|
||||
- [[frank-small-llama-swift15-iq3s-mtp-131072]] — **ACTIVE since 2026-10-05
|
||||
(evening); 2 × 131,072 no-spec since 2026-10-06** (root systemd unit,
|
||||
port 7771): llama.cpp master (CUDA sm_86) serving ukisai/Swift-1.5-
|
||||
Qwen3.8-27B-GSQ-RCO-GGUF **IQ3_S-mtp** (27B; the -mtp file's head is
|
||||
present but NOT drafted — speculation OFF by Frank's order, ngram skipped
|
||||
on the recurrent-rollback law) on the RTX 3090, TWO 131,072-token
|
||||
conversations at Frank's 128k cap (pool 262144 / parallel 2 — the tight
|
||||
fit he ordered: **1,397 MiB measured free**, the 2-3 GiB floor knowingly
|
||||
violated) with q8_0 K + q8_0 V (FP8 class, 34 KiB/token vs 64 at f16),
|
||||
ubatch 512, and vision via unsloth's mmproj-Qwen3.8-27B-F16.gguf — no
|
||||
Swift projector ships, and the base vision stack was hash-proven
|
||||
byte-identical to Swift's (6/6 tensors). Decode 48.0 t/s without the
|
||||
drafter (MTP measured 58.9–66.9; references kept in frank-small.env).
|
||||
The GSQ-RCO "IQ3_S" tier is a per-tensor allocation mix of stock types.
|
||||
Replaces BOTH same-day payloads: the live MiMo Q6_K shape (below) and
|
||||
the Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but
|
||||
never deployed
|
||||
- [[frank-small-llama-swift15-unc-iq3xxs-mtp-131072]] — **ACTIVE since
|
||||
2026-10-05 (evening); UNCECERSED RCO-IQ3_XXS with MTP back on since
|
||||
2026-10-06 evening** (root systemd unit, port 7771): llama.cpp master
|
||||
(CUDA sm_86) serving ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-
|
||||
GGUF **RCO-IQ3_XXS-mtp** (27B; the MTP head in the file is DRAFTED —
|
||||
draft-mtp depth 3, ngram skipped on the recurrent-rollback law) on the
|
||||
RTX 3090, TWO 131,072-token conversations at Frank's 128k cap (pool
|
||||
262144 / parallel 2 — the tightest fit in the endpoint's history:
|
||||
**489 MiB measured free**, the 2-3 GiB floor knowingly violated) with
|
||||
q8_0 K + q8_0 V (FP8 class, 34 KiB/token vs 64 at f16), ubatch 512, and
|
||||
vision through the publisher's own mmproj-BF16.gguf. Decode 63.3 t/s
|
||||
recipe / 72.9 temp 0 (MTP acceptance 49.8–62.4 %; 48.0 no-spec). The
|
||||
RCO-IQ3_XXS tier is a per-tensor allocation mix of stock types; the
|
||||
payload is the UNCECERSED Swift 1.5 (23/100 refusals vs Swift 1.5's
|
||||
98/100). Replaces BOTH same-day payloads: the live MiMo Q6_K shape
|
||||
(below) and the Ornith-1.5-9B Q8_0 swap that was committed
|
||||
(77decb5..380aafa) but never deployed
|
||||
- [[frank-small-llama-mimo-q6k-2x200704]] — **RETIRED 2026-10-05 (evening;
|
||||
rollback target; payload on disk)**: llama.cpp master (CUDA sm_86)
|
||||
serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K**, TWO slots of
|
||||
|
||||
Reference in new issue
Block a user