feat(frank-zealot): Bonsai-2-9B PQ2_0 with the MTP head at 3 x 262,144
Swap the 3090 endpoint from Ternary-Bonsai-2-27B PQ2_0 (2 x 200,704,
fork build 10754) to Ternary-Bonsai-2-9B PQ2_0 with the MTP head IN the
file, three conversations at the trained 262,144 window
(--ctx-size 786432 --parallel 3), on vendor fork build 10777 -- the
minimum build that pairs MTP with a quantized V cache. q8_0 K + q8_0 V
is the only KV that fits: 13,056 MiB against a 26,112 MiB f16 pair.
Settled boot 17,876 used / 6,249 free.
MTP is armed and then DECLINED by measurement: the draft context bills
its own f16 KV over the whole pool (3,072 MiB) and at UBATCH_SIZE=2048
that halves decode (167.2 -> 109.2 t/s). The documented MTP arm
(SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024, 213.5 t/s) is recorded in the
env header rather than left to rediscovery.
Two bugs the trial caught: ${SPEC_PRESENCE_PENALTY:=0.0} was a
default-assignment, so the env value always won and the no-presence-penalty
MTP arm was unreachable (now ${VAR=0.0}); and the Q8_0 mmproj is the
publisher's requirement for concurrent image requests, which matters at
--parallel 3.
contextWindow is now 262144 -- the widest in the fleet -- so :7771 is the
compaction hub. 27B runbook re-banned RETIRED with its bench-window
section closed; indexes, homepage card, OpenWebUI doc and the worklog
re-keyed.
This commit is contained in:
1 parent
2b127163c8
commit
466fd477d8
14 files changed
+1001
-314
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork), Bonsai-2-27B PQ2_0, 2 x 200704, q8_0 KV (ACTIVE)"
|
||||
description: "The ACTIVE frank-zealot runbook (since 2026-10-17) — the PrismML fork build of llama.cpp serving prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 on the RTX 3090: two conversations of 200,704 tokens with q8_0 K + q8_0 V (f16 KV does not fit this card at this window), vision through the publisher's Q8_0 Qwen3-VL projector, MTP impossible (no tensors in the file), and the measured four-arm VRAM census that priced the shape."
|
||||
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10754), Bonsai-2-27B PQ2_0, 2 x 200704, q8_0 KV (RETIRED, rollback target)"
|
||||
description: "The frank-zealot runbook of 2026-10-17 → 2026-10-18 (RETIRED — rollback target) — the PrismML fork build 10754 of llama.cpp serving prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 on the RTX 3090: two conversations of 200,704 tokens with q8_0 K + q8_0 V (f16 KV does not fit this card at this window), vision through the publisher's Q8_0 Qwen3-VL projector, MTP impossible (no tensors in the file), and the measured four-arm VRAM census that priced the shape. Superseded by the 9B MTP export at 3 × 262,144."
|
||||
tags:
|
||||
- llm
|
||||
- frank-zealot
|
||||
@@ -14,12 +14,41 @@ timestamp: 2026-10-17
|
||||
|
||||
# frank-zealot — llama.cpp (PrismML) Bonsai-2-27B PQ2_0, 2 × 200,704, q8_0 KV
|
||||
|
||||
Live since 2026-10-17, replacing the
|
||||
[[frank-zealot-llama-qwen35-9b-q6k-2x200704]] era (Qwen3.5-9B Q6_K, exact f16
|
||||
KV, same 2 × 200,704 shape). The trade: the ternary 27B costs ~3.4× the weights
|
||||
of the 9B and buys a 3× larger, stronger base at the SAME window contract —
|
||||
paid for by quantizing KV (f16 KV is arithmetically impossible for this
|
||||
payload on this card, measured below).
|
||||
**RETIRED 2026-10-17 → 2026-10-18** (AGENTS.md Runbook Index), superseded by
|
||||
[[frank-zealot-llama-prism-bonsai9-mtp]] — prism-ml/Ternary-Bonsai-2-**9B**
|
||||
PQ2_0 **with the MTP head**, THREE conversations at the native 262,144 window
|
||||
on fork build 10777. This era stays the **rollback target**: the payload
|
||||
(`/home/dank/models/Bonsai2-PQ2_0/`) and the 10754 fork tree
|
||||
(`~/GitPackages/llama-prism/`) are both still on disk. It owns the laws below —
|
||||
the 34,816 B/token KV figure over 16 full-attention layers, the 25,088 MiB f16
|
||||
OOM wall, and the two-boot VRAM pricing that every later shape on this card
|
||||
reused.
|
||||
|
||||
## 2026-10-18 bench window — production unit stopped, candidate arm on 7771
|
||||
|
||||
As of 2026-10-18 frank-zealot sits in a deliberate, dated **bench window**
|
||||
(fleet Benchmarking Policy: stop the production unit, bench candidate boots
|
||||
against the service itself). The journal shows the stop at 03:38:31 and the
|
||||
known llama.cpp shutdown shape — SIGTERM ignored past `TimeoutStopSec`,
|
||||
`State 'stop-sigterm' timed out. Killing.` at 03:40:01 — after which the unit
|
||||
reports `failed` (`is-enabled` → enabled); self-healing is merely parked
|
||||
while the window runs. What answers 7771 is a **hand-booted candidate arm**
|
||||
(`llama-prism-b10777/llama-server` on the 9B MTP file
|
||||
`Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf`, alias
|
||||
`frank-zealot`), not this runbook's payload: candidate-arm flags are not a
|
||||
state — the pid and shape moved between two censuses ten minutes apart (one
|
||||
argv read `--ctx-size 786432 --parallel 3 --spec-type draft-mtp`; a later
|
||||
`/props` read `n_ctx 262144`, `speculative.types=none`); quote neither, and
|
||||
expect `/v1/models` to report the candidate until the window closes. The
|
||||
window's remedy is the redeploy: `./scripts/deploy.sh dankclaw --service
|
||||
frank-zealot`. The production trio below is unchanged — the last PRODUCTION
|
||||
boot under the unit (03:14–03:38 the same morning) printed the contract
|
||||
exactly: `n_ctx = 401408`, `n_ctx_seq = 200704`, `CUDA0 KV buffer size =
|
||||
13328.00 MiB`, 2 slots (journal).
|
||||
|
||||
**Window CLOSED 2026-10-18:** that candidate became production. The redeploy
|
||||
shipped the 9B MTP export at `3 × 262,144` on fork build 10777 — see
|
||||
[[frank-zealot-llama-prism-bonsai9-mtp]]; this payload is the rollback target.
|
||||
|
||||
## Identity
|
||||
|
||||
|
||||
Reference in new issue
Block a user