---
type: reference
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10754), Bonsai-2-27B PQ2_0, 2 x 200704, q8_0 KV (RETIRED, rollback target)"
description: "The frank-zealot runbook of 2026-10-17 → 2026-10-18 (RETIRED — rollback target) — the PrismML fork build 10754 of llama.cpp serving prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 on the RTX 3090: two conversations of 200,704 tokens with q8_0 K + q8_0 V (f16 KV does not fit this card at this window), vision through the publisher's Q8_0 Qwen3-VL projector, MTP impossible (no tensors in the file), and the measured four-arm VRAM census that priced the shape. Superseded by the 9B MTP export at 3 × 262,144."
tags:
  - llm
  - frank-zealot
  - dankclaw
  - llama-cpp
  - gguf
  - rtx3090
timestamp: 2026-10-17
---

# frank-zealot — llama.cpp (PrismML) Bonsai-2-27B PQ2_0, 2 × 200,704, q8_0 KV

**RETIRED 2026-10-17 → 2026-10-18** (AGENTS.md Runbook Index), superseded by
[[frank-zealot-llama-prism-bonsai9-mtp]] — prism-ml/Ternary-Bonsai-2-**9B**
PQ2_0 **with the MTP head**, THREE conversations at the native 262,144 window
on fork build 10777. This era stays the **rollback target**: the payload
(`/home/dank/models/Bonsai2-PQ2_0/`) and the 10754 fork tree
(`~/GitPackages/llama-prism/`) are both still on disk. It owns the laws below —
the 34,816 B/token KV figure over 16 full-attention layers, the 25,088 MiB f16
OOM wall, and the two-boot VRAM pricing that every later shape on this card
reused.

## 2026-10-18 bench window — production unit stopped, candidate arm on 7771

As of 2026-10-18 frank-zealot sits in a deliberate, dated **bench window**
(fleet Benchmarking Policy: stop the production unit, bench candidate boots
against the service itself). The journal shows the stop at 03:38:31 and the
known llama.cpp shutdown shape — SIGTERM ignored past `TimeoutStopSec`,
`State 'stop-sigterm' timed out. Killing.` at 03:40:01 — after which the unit
reports `failed` (`is-enabled` → enabled); self-healing is merely parked
while the window runs. What answers 7771 is a **hand-booted candidate arm**
(`llama-prism-b10777/llama-server` on the 9B MTP file
`Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf`, alias
`frank-zealot`), not this runbook's payload: candidate-arm flags are not a
state — the pid and shape moved between two censuses ten minutes apart (one
argv read `--ctx-size 786432 --parallel 3 --spec-type draft-mtp`; a later
`/props` read `n_ctx 262144`, `speculative.types=none`); quote neither, and
expect `/v1/models` to report the candidate until the window closes. The
window's remedy is the redeploy: `./scripts/deploy.sh dankclaw --service
frank-zealot`. The production trio below is unchanged — the last PRODUCTION
boot under the unit (03:14–03:38 the same morning) printed the contract
exactly: `n_ctx = 401408`, `n_ctx_seq = 200704`, `CUDA0 KV buffer size =
13328.00 MiB`, 2 slots (journal).

**Window CLOSED 2026-10-18:** that candidate became production. The redeploy
shipped the 9B MTP export at `3 × 262,144` on fork build 10777 — see
[[frank-zealot-llama-prism-bonsai9-mtp]]; this payload is the rollback target.

## Identity

| | |
|---|---|
| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB (24,123 MiB visible to the engine) |
| Endpoint | `http://192.168.0.143:7771` — served model id **`frank-zealot`** (`ALIAS`), `/v1` OpenAI surface |
| Runtime | **PrismML fork build** `/home/dank/GitPackages/llama-prism/llama-server` — vendor release, `--version` = `0.2.0-dev (build 10754, commit 2459f68b5)`, sha256 `12ae5baba033d8cd…`. NOT a preference: see *Why the fork* |
| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
| Weights | `/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-PQ2_0.gguf` — 7,206,168,928 B, sha256 `3907dc1658db1f78a9826bf8d5bcb8dc65db0d466388937af57f2294fae62ec1` |
| Projector | `Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf` — 629,246,976 B (full sha256 pinned in the fetch script), Qwen3-VL tower; **873.10 MiB worst-case VRAM, loaded AFTER the engine's fit check** |
| Fetch | [`scripts/download-frank-zealot-bonsai2.sh`](../../scripts/download-frank-zealot-bonsai2.sh) — `-f`, byte-count + sha256 verified, idempotent, `--verify-only`, no token on the command line |
| Seam | `tests/frank_zealot_unit.bats` (argv assertions against a stub engine) |

## Why the fork — the format is the lock

Read from the file's own metadata, not the model card:

|Key|Value|
|---|---|
|`general.architecture`|`qwen35` (hybrid: Gated DeltaNet + softmax attention)|
|`qwen35.block_count`|64, `full_attention_interval` 4 ⇒ **16 full-attention + 48 DeltaNet layers**|
|Attention|24 heads / **4 KV heads** × key 256 / value 256; `rope.dimension_sections` [11,11,10,0], `freq_base` 1e7, partial rope `dimension_count` 64|
|Widths|embedding 5,120, feed-forward 17,408|
|`qwen35.ssm.*`|conv_kernel 4, state_size 128, group_count 16, **time_step_rank 48**, **inner_size 6,144**|
|Window / vocab|`context_length` 262,144; vocab 248,320 (gpt2 BPE, pre `qwen35`), eos 248046|
|`general.file_type`|**141 = PQ2_0 — 2.13 bpw (group 128)**|
|Tensor histogram|851 tensors: **`pq2_0` ×402** (custom GGML type **142**), f32 ×353, bf16 ×96|
|`prism.hadamard.*`|transform `normalized-sylvester-walsh-hadamard`, block_size 1024, axis `input-last-dimension`, sign_mode explicit, **401 named weights**, sign widths [5120, 6144, 17408], 28,672 sign values, inverse applied on `token_embd.weight`, `gdn_v_grouped` true|
|Sampler metadata|`general.sampling.temp` 1.0 / `top_p` 0.95 / `top_k` 20|
|Chat template|8,952 chars; reads `enable_thinking`, **`reasoning_effort` ∈ {`xhigh` (default), `medium`, `low` — anything else is `raise_exception`**, `preserve_thinking`, `tools`/`tool_call`, image + video content parts|

Stock llama.cpp cannot load this file: type 142 is not in its enum (`gguf-py`'s
stock `GGMLQuantizationType` raises `ValueError: 142 is not a valid…` on the
same read), and even a loader that parsed it would ignore the
`prism.hadamard.*` dequant transform the weights depend on. Same class of lock
as the ROCmFPX custom-type era. The vendor build 10754 already carries the
September upstream fixes (AVX-512 load segfault, and a `GGML_CUDA_FA_ALL_QUANTS`
-capable CUDA backend — proven by q8_0 V serving under flash attention at the
correct 34,816 B/token, below).

## VRAM census (2026-10-17, four arms, production stopped)

The engine's own `common_memory_breakdown_print` ledger + settled `nvidia-smi`,
all arms `--n-gpu-layers 99 --flash-attn on --batch-size 4096 --ubatch-size 2048`,
mmproj attached:

|Shape|KV|projected `model + context + compute`|settled used / free|verdict|
|---|---|---|---|---|
|**2 × 200,704 (pool 401,408)**|**q8_0 / q8_0**|21,439 = 6,539 + 13,627 + 1,272|**22,664 / 1,461**|**DEPLOYED**|
|2 × 200,704|f16 / f16|33,198 = 6,539 + 25,387 + 1,272|—|`cudaMalloc failed: out of memory` on the **25,088.00 MiB** KV request|
|1 × 262,144|q8_0 / q8_0|16,905 = 6,539 + 8,853 + 1,512|18,130 / 5,995|fits, but one slot serializes compaction|
|2 × 262,144|q8_0 / q8_0|≈25,758 (arithmetic, same terms)|—|does not fit (f16 analogue measured 41,118)|
|2 × 200,704|q4_0 / q4_0|15,167 = 6,539 + 7,355 + 1,272|16,392 / 7,733|the roomy fallback rung|

**KV rate law (confirmed to the MiB):** 16 full layers × 4 KV heads × 256 dim =
16,384 elements/token/cache; q8_0 bills 17,408 B per cache ⇒ **34,816 B/token**,
so the 401,408 pool = **13,328.00 MiB** — the ledger printed exactly that
(`CUDA0 KV buffer size = 13328.00 MiB`). f16 doubles it to 25,088 MiB, which is
the wall. The 48 DeltaNet layers bill a fixed recurrent state per **slot**:
context 13,627 − KV 13,328 = **299 MiB for two slots (~150 MiB each)** — pool
arithmetic alone under-prices the shape by that much.

**The projector is outside the fit check.** `[mtmd] estimated worst-case memory
usage of mmproj is 873.10 MiB` is printed before fitting but not billed in it:
settled used (22,664) − projection (21,439) ≈ 1.2 GiB is that projector plus the
CUDA context. This is the difference between "2.6 GiB free on paper" and the
real **1.4 GiB** — the same late-allocation failure class that killed the
DSpark drafter boot in the llama.cpp eras. Any resize must re-read the settled
`nvidia-smi`, never the projection.

## Measured rates (greedy, `Server-Timing`-free `/completion` `timings`)

|Probe|Result|
|---|---|
|Short, warm, single stream (128 tok)|**76.6 t/s** decode (the 9B Q6_K predecessor ran 94.4 — the ternary 27B is ~19 % slower, weight bandwidth + Hadamard dequant)|
|First request after boot|2.6 t/s prefill / 37.7 t/s decode — **warmup artifact; never report the first request** (measured twice on this payload)|
|Prefill at ~35k depth|**1,258.4 t/s** (27.8 s)|
|Prefill at 166,027-token depth|**691.7 t/s** (240.0 s), zero errors, slot intact|
|Two concurrent streams, 256 tok each|**54.7 + 54.7 = 109.4 t/s aggregate** — `--parallel 2` really batches|
|Long-context retrieval (25,960-token prompt, needle at position 180 of 300 passages)|exact: *"Frank keeps the fleet in git, only through it."*|

Production boot (the 19:44 cutover deploy, same binary/flags/file): **75.8 t/s**
single-stream decode at a short prompt — within 1 % of the trial arm, so the
ladder above carries to the shipped shape.

## Behavior contract (probes, not assumptions)

- **Reasoning** lands in `reasoning_content` (`--reasoning-format deepseek`) ✓.
- **Tool calls**: a `get_weather` request returns `finish_reason: tool_calls`
  with `arguments: {"city":"Seattle"}` — valid JSON, no loop, no HTTP 500. The
  fork's `KNOWN_ISSUES.md` warning about malformed/looping tool calls did not
  reproduce on build 10754 at this shape.
- **Vision**: a 512 px solid-red PNG answers **"Red"** — but ONLY with a
  thinking budget ≥ ~512 tokens. At `max_tokens: 64` the request returns
  `finish_reason: "length"` with EMPTY `content` and half a thought in
  `reasoning_content`: thinking tokens count against the budget, so a
  short-budget vision probe reads as "the model is blind". Images do reach the
  model (a 512 px image bills ~280 prompt tokens: 322 total vs 42 for the text
  alone). Keep the ≥256 px probe rule from the earlier eras.
- **Log noise law**: `/completion` (text context) at `VERBOSITY=4` prints
  `failed to parse reasoning content (response_format json_schema is currently
  only supported with parse_reasoning=true), content will be stored in
  'content' field` (`probe_request.cpp:789`) whenever the model closes a raw
  prompt with ``. It is **cosmetic** — the full chain of thought
  is in `content`. The OpenAI `/v1/chat/completions` path parses it into
  `reasoning_content` correctly, which is the path the harness and every
  consumer uses. Do not diagnose this as a fault, and do not drop verbosity
  below 4 — the KV/buffer ledger in the journal is what VRAM forensics read.
- **Long raw-prompt completions can come back empty**: at ~26k+ tokens a bare
  `/completion` request generated 0–1 tokens with no error (the model emits an
  end-of-turn marker immediately, and a text context has no role frame to keep
  it talking). Chat traffic is unaffected; nothing in the harness posts raw
  `/completion` at depth.

## Ops notes

- **Trial-window law (cost one boot window on 2026-10-17):** a scratch env file
  must use the serve script's OWN variable names — `CACHE_TYPE_K`/`CACHE_TYPE_V`,
  `BIND_ADDR`, `NGL`. An invented `KV_CACHE_TYPE` is silently swallowed by the
  script's `: "${CACHE_TYPE_K:=f16}"` default and the arm boots **f16**, which
  for this payload is an OOM that looks like a model problem.
- **`/metrics` needs `--metrics` (fixed 2026-10-18):** the fork answers **501 Not
  Implemented** without the flag, and Dankclaw's Prometheus scrapes `/metrics`, so
  `up{job="frank-zealot"}` has been 0 for the whole retained metric history — the
  oldest sample (2026-10-11) is already 0 — while the endpoint itself answered
  `/health` 200 the entire time. `METRICS=1` in `scripts/frank-zealot.env` now
  ships the flag and the seam asserts it reaches the argv. halogen serves
  `/metrics` natively, which is why that target was always green.
- **Base alignment:** Bonsai-2 is the BASE (censored) Qwen3.8-27B lineage, so a
  refusal is possible on a compaction or subagent request. The CRACK re-export
  (`dealignai/Bonsai-2-27B-Ternary-CRACK-GGUF`, already on disk at
  `/home/dank/models/Bonsai2-CRACK/Bonsai-2-27B-PQ2_0-CRACK.gguf`, sha256
  `5b24ea3e…`) is a **one-line `MODEL_FILE` swap**: the header census proves
  byte-identical geometry — 851 tensors, the same `{142:402, f32:353, bf16:96}`
  histogram, 64 blocks, vocab 248,320, the same 8,952-char template. The VRAM
  ledger does not move. If a refusal ever surfaces, that is the fix; there is
  also an MTP-bearing CRACK re-export upstream
  (`ManniX-ITA/…-CRACK-MTP-GGUF`, 7,657,489,920 B, +451 MB head) if decode ever
  needs the drafter — untested here.
- **Compaction hub unchanged:** all three endpoints still route
  `compactionModel` here; 200,704 per slot is the fleet's only window that fits
  a worst-case compaction input, and `--parallel 2` keeps compaction off the
  critical path of a live generation.
- **Harness side** (`~/.omp/agent/models.yml`): `contextWindow: 200704` and
  `maxTokens: 16384` unchanged; `extraBody` now quotes the payload's own
  1.0 / 0.95 / 20 with `preserve_thinking: true`. The `reasoningEffortMap` fold
  became **load-bearing** (the template raises on any level outside
  `xhigh|medium|low`) — an unmapped level is a hard 500 on the role's first
  request, naming Jinja rather than the model.
- **Rollback:** revert the swap commit. The Qwen3.5-9B Q6_K payload, its mmproj
  and the master CUDA build all remain on disk
  (`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`,
  `~/GitPackages/llama-master/build-cuda/`), and the retired era keeps its own
  page: [[frank-zealot-llama-qwen35-9b-q6k-2x200704]].
- **Deployed 2026-10-17 19:44** (`deploy.sh dankclaw --service frank-zealot
  --service homepage`): boot ledger 22,664 used / 1,461 free, engine fit
  projected 21,439 and left 2,119 ≥ 1,872 MiB, KV 13,328.00 MiB at
  `n_ctx_seq = 200,704`, `n_slots = 2`, projector loaded; live probes green
  (`/v1/models` serves `frank-zealot`, exact `pong` + `reasoning_content`,
  `tool_calls` `{"city":"Seattle"}`, 512 px PNG → **Red**). e2e gate 33/33.
- **Picker row is live state:** the OpenWebUI `model` row's
  `meta.description` named the 9B, so it was re-keyed in the database too
  (backup `webui.db.bak-20261017a`). The row was written WITHOUT restarting
  `open-webui` — the cached list refreshes on its next restart, and forcing one
  would have logged every family session out for a cosmetic string. See
  [[OpenWebUI]] for the picker trio.

## Related

- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — the replaced era (rollback target)
- [[frank-dragoon-flash-server-v2-swift15]] — the abliterated sibling on the iGPU
- [[frank-archon-radiance-qwen38-iq4r]] — the censored-base flagship this payload shares base alignment with
- [[deployment]] — deploy flow and incidents