docs: record the two-slot no-spec frank-small shape (measured 1,397 MiB free, 48.0 t/s)
Runbook reshaped: 2 x 131072 no-spec identity, slot-2 arithmetic (drafter ~0.4 GiB + ubatch ~0.3 GiB bought the second 4,352 MiB KV slot), ngram-mod skipped on the recurrent-rollback law, MTP references kept (44.8-52.1% acceptance, 58.9-66.9 t/s), rollback = one revert of 812d9eb. AGENTS fleet row + section, LLM/index.md intro + entry, root index entry all flipped to the 2026-10-06 shape. 360/360 suite green; e2e gate 33/33 green at deploy.
This commit is contained in:
@@ -16,9 +16,10 @@ ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
|
||||
engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
|
||||
lives on frank-small)** and
|
||||
**llama.cpp CUDA (frank-small, port 7771: ukisai Swift-1.5-Qwen3.8-27B
|
||||
GSQ-RCO IQ3_S-mtp GGUF — 27B with the MTP head IN the file, ONE 131,072
|
||||
conversation at Frank's 128k cap, q8_0 K + q8_0 V (FP8 class), vision via
|
||||
the base-model projector, on the RTX 3090)**.
|
||||
GSQ-RCO IQ3_S-mtp GGUF — 27B (head present, NOT drafted), TWO 131,072
|
||||
conversations at Frank's 128k cap (tight fit, spec off since 2026-10-06),
|
||||
q8_0 K + q8_0 V (FP8 class), vision via the base-model projector, on the
|
||||
RTX 3090)**.
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -81,14 +82,18 @@ frank-small, port 7771]
|
||||
Rollback: revert the swap commit; the ROCmFPX payload + fork build +
|
||||
`build-rocmfpx.sh` pin all remain on disk
|
||||
- [[frank-small-llama-swift15-iq3s-mtp-131072]] — **ACTIVE since 2026-10-05
|
||||
(evening)** (root systemd unit, port 7771): llama.cpp master (CUDA sm_86)
|
||||
serving ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF **IQ3_S-mtp** (27B, the
|
||||
checkpoint's own MTP head IN the file — `--spec-type draft-mtp`, depth 3)
|
||||
on the RTX 3090, ONE 131,072-token conversation at Frank's 128k cap (a
|
||||
second 131,072 slot misses the 2-3 GiB floor) with q8_0 K + q8_0 V (FP8
|
||||
class, 34 KiB/token vs 64 at f16) and vision via unsloth's
|
||||
mmproj-Qwen3.8-27B-F16.gguf — no Swift projector ships, and the base
|
||||
vision stack was hash-proven byte-identical to Swift's (6/6 tensors).
|
||||
(evening); 2 × 131,072 no-spec since 2026-10-06** (root systemd unit,
|
||||
port 7771): llama.cpp master (CUDA sm_86) serving ukisai/Swift-1.5-
|
||||
Qwen3.8-27B-GSQ-RCO-GGUF **IQ3_S-mtp** (27B; the -mtp file's head is
|
||||
present but NOT drafted — speculation OFF by Frank's order, ngram skipped
|
||||
on the recurrent-rollback law) on the RTX 3090, TWO 131,072-token
|
||||
conversations at Frank's 128k cap (pool 262144 / parallel 2 — the tight
|
||||
fit he ordered: **1,397 MiB measured free**, the 2-3 GiB floor knowingly
|
||||
violated) with q8_0 K + q8_0 V (FP8 class, 34 KiB/token vs 64 at f16),
|
||||
ubatch 512, and vision via unsloth's mmproj-Qwen3.8-27B-F16.gguf — no
|
||||
Swift projector ships, and the base vision stack was hash-proven
|
||||
byte-identical to Swift's (6/6 tensors). Decode 48.0 t/s without the
|
||||
drafter (MTP measured 58.9–66.9; references kept in frank-small.env).
|
||||
The GSQ-RCO "IQ3_S" tier is a per-tensor allocation mix of stock types.
|
||||
Replaces BOTH same-day payloads: the live MiMo Q6_K shape (below) and
|
||||
the Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but
|
||||
|
||||
Reference in New Issue
Block a user