feat(frank-zealot): Bonsai-2-9B PQ2_0 with the MTP head at 3 x 262,144

Swap the 3090 endpoint from Ternary-Bonsai-2-27B PQ2_0 (2 x 200,704,
fork build 10754) to Ternary-Bonsai-2-9B PQ2_0 with the MTP head IN the
file, three conversations at the trained 262,144 window
(--ctx-size 786432 --parallel 3), on vendor fork build 10777 -- the
minimum build that pairs MTP with a quantized V cache. q8_0 K + q8_0 V
is the only KV that fits: 13,056 MiB against a 26,112 MiB f16 pair.
Settled boot 17,876 used / 6,249 free.

MTP is armed and then DECLINED by measurement: the draft context bills
its own f16 KV over the whole pool (3,072 MiB) and at UBATCH_SIZE=2048
that halves decode (167.2 -> 109.2 t/s). The documented MTP arm
(SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024, 213.5 t/s) is recorded in the
env header rather than left to rediscovery.

Two bugs the trial caught: ${SPEC_PRESENCE_PENALTY:=0.0} was a
default-assignment, so the env value always won and the no-presence-penalty
MTP arm was unreachable (now ${VAR=0.0}); and the Q8_0 mmproj is the
publisher's requirement for concurrent image requests, which matters at
--parallel 3.

contextWindow is now 262144 -- the widest in the fleet -- so :7771 is the
compaction hub. 27B runbook re-banned RETIRED with its bench-window
section closed; indexes, homepage card, OpenWebUI doc and the worklog
re-keyed.
This commit is contained in:
Frank committed 2026-10-18 13:15:18 -07:00
1 parent 2b127163c8
commit 466fd477d8
14 files changed
+1001 -314

No files matched your search

@@ -1,7 +1,7 @@
---
type: reference
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork), Bonsai-2-27B PQ2_0, 2 x 200704, q8_0 KV (ACTIVE)"
description: "The ACTIVE frank-zealot runbook (since 2026-10-17) — the PrismML fork build of llama.cpp serving prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 on the RTX 3090: two conversations of 200,704 tokens with q8_0 K + q8_0 V (f16 KV does not fit this card at this window), vision through the publisher's Q8_0 Qwen3-VL projector, MTP impossible (no tensors in the file), and the measured four-arm VRAM census that priced the shape."
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10754), Bonsai-2-27B PQ2_0, 2 x 200704, q8_0 KV (RETIRED, rollback target)"
description: "The frank-zealot runbook of 2026-10-17 → 2026-10-18 (RETIRED — rollback target) — the PrismML fork build 10754 of llama.cpp serving prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 on the RTX 3090: two conversations of 200,704 tokens with q8_0 K + q8_0 V (f16 KV does not fit this card at this window), vision through the publisher's Q8_0 Qwen3-VL projector, MTP impossible (no tensors in the file), and the measured four-arm VRAM census that priced the shape. Superseded by the 9B MTP export at 3 × 262,144."
tags:
- llm
- frank-zealot
@@ -14,12 +14,41 @@ timestamp: 2026-10-17
# frank-zealot — llama.cpp (PrismML) Bonsai-2-27B PQ2_0, 2 × 200,704, q8_0 KV
Live since 2026-10-17, replacing the
[[frank-zealot-llama-qwen35-9b-q6k-2x200704]] era (Qwen3.5-9B Q6_K, exact f16
KV, same 2 × 200,704 shape). The trade: the ternary 27B costs ~3.4× the weights
of the 9B and buys a 3× larger, stronger base at the SAME window contract —
paid for by quantizing KV (f16 KV is arithmetically impossible for this
payload on this card, measured below).
**RETIRED 2026-10-17 → 2026-10-18** (AGENTS.md Runbook Index), superseded by
[[frank-zealot-llama-prism-bonsai9-mtp]] — prism-ml/Ternary-Bonsai-2-**9B**
PQ2_0 **with the MTP head**, THREE conversations at the native 262,144 window
on fork build 10777. This era stays the **rollback target**: the payload
(`/home/dank/models/Bonsai2-PQ2_0/`) and the 10754 fork tree
(`~/GitPackages/llama-prism/`) are both still on disk. It owns the laws below —
the 34,816 B/token KV figure over 16 full-attention layers, the 25,088 MiB f16
OOM wall, and the two-boot VRAM pricing that every later shape on this card
reused.
## 2026-10-18 bench window — production unit stopped, candidate arm on 7771
As of 2026-10-18 frank-zealot sits in a deliberate, dated **bench window**
(fleet Benchmarking Policy: stop the production unit, bench candidate boots
against the service itself). The journal shows the stop at 03:38:31 and the
known llama.cpp shutdown shape — SIGTERM ignored past `TimeoutStopSec`,
`State 'stop-sigterm' timed out. Killing.` at 03:40:01 — after which the unit
reports `failed` (`is-enabled` → enabled); self-healing is merely parked
while the window runs. What answers 7771 is a **hand-booted candidate arm**
(`llama-prism-b10777/llama-server` on the 9B MTP file
`Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf`, alias
`frank-zealot`), not this runbook's payload: candidate-arm flags are not a
state — the pid and shape moved between two censuses ten minutes apart (one
argv read `--ctx-size 786432 --parallel 3 --spec-type draft-mtp`; a later
`/props` read `n_ctx 262144`, `speculative.types=none`); quote neither, and
expect `/v1/models` to report the candidate until the window closes. The
window's remedy is the redeploy: `./scripts/deploy.sh dankclaw --service
frank-zealot`. The production trio below is unchanged — the last PRODUCTION
boot under the unit (03:14–03:38 the same morning) printed the contract
exactly: `n_ctx = 401408`, `n_ctx_seq = 200704`, `CUDA0 KV buffer size =
13328.00 MiB`, 2 slots (journal).
**Window CLOSED 2026-10-18:** that candidate became production. The redeploy
shipped the 9B MTP export at `3 × 262,144` on fork build 10777 — see
[[frank-zealot-llama-prism-bonsai9-mtp]]; this payload is the rollback target.
## Identity