docs: record the two-slot no-spec frank-small shape (measured 1,397 MiB free, 48.0 t/s)

Runbook reshaped: 2 x 131072 no-spec identity, slot-2 arithmetic (drafter ~0.4 GiB + ubatch ~0.3 GiB bought the second 4,352 MiB KV slot), ngram-mod skipped on the recurrent-rollback law, MTP references kept (44.8-52.1% acceptance, 58.9-66.9 t/s), rollback = one revert of 812d9eb. AGENTS fleet row + section, LLM/index.md intro + entry, root index entry all flipped to the 2026-10-06 shape. 360/360 suite green; e2e gate 33/33 green at deploy.
This commit is contained in:
Frank
2026-10-06 00:15:55 -07:00
parent 812d9ebff5
commit 5e166543fa
4 changed files with 134 additions and 80 deletions
+16 -11
View File
@@ -16,9 +16,10 @@ ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
lives on frank-small)** and
**llama.cpp CUDA (frank-small, port 7771: ukisai Swift-1.5-Qwen3.8-27B
GSQ-RCO IQ3_S-mtp GGUF — 27B with the MTP head IN the file, ONE 131,072
conversation at Frank's 128k cap, q8_0 K + q8_0 V (FP8 class), vision via
the base-model projector, on the RTX 3090)**.
GSQ-RCO IQ3_S-mtp GGUF — 27B (head present, NOT drafted), TWO 131,072
conversations at Frank's 128k cap (tight fit, spec off since 2026-10-06),
q8_0 K + q8_0 V (FP8 class), vision via the base-model projector, on the
RTX 3090)**.
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -81,14 +82,18 @@ frank-small, port 7771]
Rollback: revert the swap commit; the ROCmFPX payload + fork build +
`build-rocmfpx.sh` pin all remain on disk
- [[frank-small-llama-swift15-iq3s-mtp-131072]] — **ACTIVE since 2026-10-05
(evening)** (root systemd unit, port 7771): llama.cpp master (CUDA sm_86)
serving ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF **IQ3_S-mtp** (27B, the
checkpoint's own MTP head IN the file — `--spec-type draft-mtp`, depth 3)
on the RTX 3090, ONE 131,072-token conversation at Frank's 128k cap (a
second 131,072 slot misses the 2-3 GiB floor) with q8_0 K + q8_0 V (FP8
class, 34 KiB/token vs 64 at f16) and vision via unsloth's
mmproj-Qwen3.8-27B-F16.gguf — no Swift projector ships, and the base
vision stack was hash-proven byte-identical to Swift's (6/6 tensors).
(evening); 2 × 131,072 no-spec since 2026-10-06** (root systemd unit,
port 7771): llama.cpp master (CUDA sm_86) serving ukisai/Swift-1.5-
Qwen3.8-27B-GSQ-RCO-GGUF **IQ3_S-mtp** (27B; the -mtp file's head is
present but NOT drafted — speculation OFF by Frank's order, ngram skipped
on the recurrent-rollback law) on the RTX 3090, TWO 131,072-token
conversations at Frank's 128k cap (pool 262144 / parallel 2 — the tight
fit he ordered: **1,397 MiB measured free**, the 2-3 GiB floor knowingly
violated) with q8_0 K + q8_0 V (FP8 class, 34 KiB/token vs 64 at f16),
ubatch 512, and vision via unsloth's mmproj-Qwen3.8-27B-F16.gguf — no
Swift projector ships, and the base vision stack was hash-proven
byte-identical to Swift's (6/6 tensors). Decode 48.0 t/s without the
drafter (MTP measured 58.9–66.9; references kept in frank-small.env).
The GSQ-RCO "IQ3_S" tier is a per-tensor allocation mix of stock types.
Replaces BOTH same-day payloads: the live MiMo Q6_K shape (below) and
the Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but