serverAppConfigs/serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md

---
type: reference
title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Bonsai-2-9B PQ2_0 with the MTP head, 2 × 262144, exact f16 KV (ACTIVE)"
description: "The ACTIVE frank-zealot runbook (since 2026-10-18) — the PrismML fork build 10777 of llama.cpp serving prism-ml/Ternary-Bonsai-2-9B-gguf at PQ2_0 on the RTX 3090: TWO conversations of 262,144 tokens (every slot at the model's trained window) over an UNQUANTIZED f16 K + V cache, vision through the publisher's Q8_0 Qwen3-VL projector, and the boot-arm census that put speculative decoding OFF by measurement even though the checkpoint carries the MTP head."
tags:
  - llm
  - frank-zealot
  - dankclaw
  - llama-cpp
  - gguf
  - rtx3090
  - mtp
  - spec-decode
timestamp: 2026-10-18
---

# frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0 + MTP head, 2 × 262,144, exact f16 KV

Live since 2026-10-18, replacing the
[[frank-zealot-llama-prism-bonsai2-pq20]] era (Ternary-Bonsai-2-**27B** PQ2_0 at
2 × 200,704). The original trade: the 9B bills **half** the KV per token of the
27B, and that is what funded **three** conversations at the model's **native
262,144** window where the 27B afforded only two shortened ones (Frank's ask:
*"if you can possibly fit a third context, do it"*).

The shape moved again the same evening (Frank: *"shift frank-zealot to 2
parallels of unquantized kv"*). The KV cache went **exact** — f16 K + f16 V, no
quantization error anywhere in the attention state — and the **slot count paid
for it: 3 → 2**. The per-slot contract never moved; every conversation is still
262,144, still the trained window. What the dtype bought is fidelity, and what
it costs is one concurrent conversation plus the bytes attention reads per
generated token.

## Payload identity

| Item | Value |
|---|---|
| Runtime | `/home/dank/GitPackages/llama-prism-b10777/llama-server` — PrismML **fork** build, `--version` → `0.2.0-dev (build 10777, commit e3de71522)` |
| Checkpoint | `/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf` — 2,675,410,720 B, Hub-LFS sha256 `c79fcdfaf6b7bf47089b50dadcfd11f9ce7d3e96adf3bbefde7912e5ef69e22b` |
| Vision | `/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf` — 624,229,664 B, Hub-LFS sha256 `e0cb6f6988cea323ef798cd77d019bbb819af75582f034eaab0e7232314953db` |
| Fetch | `scripts/download-frank-zealot-bonsai9.sh [DEST]` (`--verify-only`; `-f`, byte-count and LFS sha256 per file) |
| Alias | `frank-zealot` (`ALIAS`); the SERVICE/unit/scripts keep the StarCraft name, only the wire id is the model id |
| Port | 7771, root systemd unit `frank-zealot.service` (relinked), `TimeoutStopSec=180` |

### The runtime lock has two tumblers now

1. **The tensor type.** `general.file_type 141` with **202 tensors of the fork's
   PQ2_0 type** (of 442 total; the rest are f32 norms/scalars and 8 q8_0 in the
   MTP block) plus `prism.hadamard.*` metadata
   (`normalized-sylvester-walsh-hadamard`, `block_size 4096`,
   `axis input-last-dimension`, `sign_mode explicit` over 201 weight names,
   inverse on `token_embd.weight`, `gdn_v_grouped true`). Stock llama.cpp
   refuses the custom type at load; a loader that ignored the Hadamard metadata
   would dequant **wrong**, silently. Same lock class as the ROCmFPX era.
2. **The build floor.** The fork's card gates **MTP beside a quantized V cache**
   on `prism-b10776-d00e252` or newer, which is why the prebuilt moved
   **10754 → 10777** for the swap. Exact f16 KV makes that gate moot — there is
   no quantized V to pair with the drafter — but tumbler 1 pins the runtime on
   its own: stock llama.cpp cannot load this file at any flag value. The 10754
   tree stays on disk (`~/GitPackages/llama-prism/`) as the 27B rollback
   runtime. `tests/frank_zealot_unit.bats` pins both paths.

## Header census (read from the GGUF, not the card)

`general.architecture qwen35`, `general.name Ternary-Bonsai-2-9B`,
`general.size_label 9B`, `general.quantization_version 2`,
`general.file_type 141`:

| Key | Value | Note |
|---|---|---|
| `qwen35.block_count` | **33** | 32 trunk blocks + `blk.32` = the MTP block |
| `qwen35.full_attention_interval` | 4 | ⇒ **8 full-attention + 24 Gated DeltaNet** trunk layers |
| `qwen35.context_length` | 262,144 | native; the deployed slot equals it |
| `qwen35.embedding_length` / `feed_forward_length` | 4,096 / 12,288 | |
| `attention.head_count` / `head_count_kv` | 16 / **4** | |
| `attention.key_length` / `value_length` | 256 / 256 | |
| `ssm.conv_kernel` / `state_size` / `group_count` | 4 / 128 / 16 | the 27B: 4 / 128 / 16 |
| `ssm.time_step_rank` / `inner_size` | **32 / 4,096** | the 27B: 48 / 6,144 |
| `rope.freq_base` / `dimension_count` / `dimension_sections` | 1e7 / 64 / `[11,11,10,0]` | |
| `qwen35.nextn_predict_layers` | **1** | the MTP head is IN the file |
| `general.sampling.*` | temp 1.0 / top_p 0.95 / top_k 20 / min_p 0.0 / repeat 1.0 | the card recipe; llama.cpp reads it itself |

**KV accounting.** Only the 8 full-attention layers scale with context:
8 × 4 KV heads × 256 dim = **8,192 elements/token per cache** (the engine prints
`n_embd_k_gqa = n_embd_v_gqa = 1024`). **The deployed pair is exact: f16 K +
f16 V bills 32,768 B/token** (2 B/element × 8,192 elements × 2 caches), so the
524,288-position pool is **16,384 MiB**. The q8_0 pair this endpoint shipped
first billed **17,408 B/token** — 1.0625 B/element, 32 codes plus one 2-byte
scale per 32-block — the **13,056.00 MiB** its ledger printed at the 786,432
pool, exactly HALF the 27B's 34,816 B/token.

**Corrected arithmetic, and it matters.** The note that lived here and in
`frank-zealot.env` — "f16 would bill 26,112 MiB" — doubled the q8_0 figure
*including* its scale overhead, overstating exact KV by 6.25 %. Exact f16 at the
786,432 pool is **24,576 MiB** against the **24,123 MiB** the driver leaves
addressable (`nvidia-smi`'s total column reads 24,576; used + free reads 24,123
— the difference is driver reservation). The wall stands in both readings; the
honest rate is what makes the fit calculable instead of superstitious. **Halving
the POOL is what funds exact KV** — the window is already at the native ceiling
and there is no YaRN in this path.

The 24 DeltaNet layers bill a fixed recurrent state per SLOT instead: the 3-slot
ledger printed `RS buffer size = 150.75 MiB (3 cells, 32 layers, 3 seqs)` ≈
**50.25 MiB a cell**, so two slots bill 100.50. The SPEC-ARMED arms printed
452.25 MiB — MTP depth 2 bills three seqs per slot, so the drafter triples even
the recurrent state. Pool arithmetic alone under-prices the shape.

## The shape (`scripts/frank-zealot.env`)

`--ctx-size 524288 --parallel 2` (⇒ `n_ctx_slot = 262,144`), `--flash-attn on`,
`--cache-type-k f16 --cache-type-v f16`, `--batch-size 4096`,
`--ubatch-size 2048`, `--n-gpu-layers 99`, `--jinja`, `--reasoning auto`,
`--reasoning-format deepseek`, `--metrics`, `--mmproj …`, `--log-verbosity 4`,
`--presence-penalty 1.0`,
**`SPEC_TYPE=none`** (which strips `--spec-type` AND every `--spec-draft-*`).

Per-slot window is the **contract** the harness quotes: `models.yml` carries
`contextWindow: 262144`, and it did **not** move with the slot count because
llama.cpp divides `--ctx-size` across slots. It refuses a prompt longer than the
SLOT with a 400 naming the limit — never a truncation — and since this swap the
slot window equals the native window, so no YaRN and no cap is in play.
`tests/frank_zealot_unit.bats` now gates the arithmetic: the shipped dtype ×
pool plus the measured non-KV ledger terms must fit 24,123 MiB with ≥1 GiB of
slack, and the gate is proven to REFUSE the impossible arm (exact KV at the old
786,432 pool = 24,576 MiB of cache alone).

## The MTP verdict: the head is in the file and it is still OFF

The drafter is **good**: acceptance 0.798 / 0.617 per position, mean accepted
length 2.41, and the card recommends depth 2. It is OFF because the draft
context bills its own **f16** KV over the WHOLE pool — `blk.32` carries real
attention (4 KV heads × 256 × 2 caches × 2 B = **4,096 B/position** at the
default dtype). The draft dtype is itself a knob — `--spec-draft-type-k` /
`--spec-draft-type-v`, both default f16, verified against this build's `--help`
(so the "no flag exists" claim that lived here is FALSE) — and quantizing the
draft to q8_0 would bill 2,176 B/position, **1,088 MiB** at the shipped pool.
**No measured arm runs a drafter beside exact KV**; every row below is f16 draft.

The census below is the **q8_0 era**: every row is q8_0 K + q8_0 V at the
786,432 pool, where the draft bill was 3,072.00 MiB and 4,830 MiB with its
compute buffers. Booted arms, same card, same traffic, 2026-10-18
(`/completion` `timings`, greedy 512 tokens; never the first request after a
boot):

| slots | spec | ubatch | used / free | decode | 3-stream | prefill @106k |
|---|---|---|---|---|---|---|
| **3 × 262,144** | **off** | **2048** | **17,876 / 6,249** | **167.2** | **280.2** | **2,877** |
| 3 × 262,144 | on | 2048 | 22,706 / 1,419 | 109.2 | — | — |
| 3 × 262,144 | on | 1024 | 21,400 / 2,725 | 213.5 | 250.8 | 2,395 |
| 2 × 262,144 | on | 2048 | 17,178 / 6,947 | 213.2 | — | — |
| 1 × 262,144 | on | 2048 | 11,652 / 12,473 | 214.2 | — | — |

**MTP is worth +28 % single-stream decode whenever the card has room** (rows
4–5 prove the drafter itself is a win; row 2 proves the squeeze is what halves
it). The arm that ran it at three windows was `SPEC_TYPE=draft-mtp` +
`UBATCH_SIZE=1024` (row 3), paying **prefill −17 %** and **2,725 MiB of
headroom** on a card whose mmproj worst-case lands AFTER the engine's own fit
check.

**Under exact KV an f16-draft arm is priced out of this card.** The draft
context is built at the TARGET's pool, so at 524,288 positions it bills
4,096 × 524,288 = **2,048.00 MiB** plus its own buffers, on a trunk that already
projects **~21,200 MiB** of the 24,123 MiB addressable. The doors back to
speculation are **one window per conversation** (row 5's class of shape), **KV
back to q8_0** with `UBATCH_SIZE=1024`, or the **unmeasured**
`--spec-draft-type-k q8_0 --spec-draft-type-v q8_0` arm — 1,088 MiB of draft,
legal on b10777, never booted beside exact KV. Frank chose exact KV over the
drafter on 2026-10-18, and the endpoint's actual job is what made that
affordable: the fleet's **compaction hub** is prefill-bound on ~170k inputs,
prefill is weight-bandwidth bound and does not re-read the cache per generated
token — so the dtype is paid in decode and concurrency, not in the work this
service exists to do.

## Boot ledger — the q8_0 3-slot production boot (2026-10-18 13:19, under the unit, verbosity 4)

```
CUDA0 model buffer size  =  2036.71 MiB    # + 257.66 MiB CPU_Mapped (mmap)
CUDA0 KV buffer size     = 13056.00 MiB    # q8_0 K + q8_0 V, 786,432 positions
CUDA0 RS buffer size     =   150.75 MiB    # 3 cells, 32 layers, 3 seqs
CUDA0 compute buffer     =  1409.02 MiB    # + 1088.87 MiB CUDA_Host, 2.84 output
mmproj worst-case (est.) =   868.32 MiB    # 248.10 CUDA + 24.93 CPU, AFTER fit
nvidia-smi settled: 17,922 MiB used / 6,203 MiB free of 24,123
n_slots = 3, n_ctx_slot = 262144, kv_unified = false
```

No clamp line, no `cudaMalloc` error. The engine's own projection (21,439-class
numbers from the 27B era) excludes the projector — **read settled `nvidia-smi`,
never the projection**.

**The exact-KV boot, measured** (2026-10-18 evening, PR #21, under the unit,
verbosity 4 — the live shape):

```
CUDA0 model buffer size  =  2036.71 MiB    # + 257.66 MiB CPU_Mapped (mmap)
CUDA0 KV buffer size     = 16384.00 MiB    # f16 K + f16 V, 524,288 positions
CUDA0 RS buffer size     =   100.50 MiB    # 2 cells, 32 layers, 2 seqs
CUDA0 compute buffer     =  1408.06 MiB    # + 1088.07 MiB CUDA_Host
mmproj worst-case (est.) =   868.32 MiB    # 248.10 CUDA + 24.93 CPU, AFTER fit
nvidia-smi settled: 21,176 MiB used / 2,949 MiB free of 24,123
n_slots = 2, n_ctx_slot = 262144, kv_unified = false
```

The pre-flight projection was right to the MiB, because the two terms it moved
are exact arithmetic: **16,384.00 MiB of KV is precisely 32,768 B/token ×
524,288 positions** — rate derivation and ledger agree with no slop, which is
what buries the old 26,112 MiB figure for good — and the recurrent state is one
50.25 MiB cell lighter (100.50 exactly). Settled card state landed at **21,176
used / 2,949 free**, within 24 MiB of the 21,200 / 2,920 projection,
`DropInPaths` empty, no clamp line, no `cudaMalloc` error. It is a desktop
margin, though: the mmproj's 868 MiB worst case is still charged AFTER the
engine's own fit check, so ~2 GiB is what an image storm has to live inside.

## Measured — the q8_0 3-slot boot (`/completion` `timings`, greedy, `ignore_eos`)

- **168.7 t/s** decode — three runs of 512 tokens: 168.4 / 168.7 / 168.7
  (the trial arm printed 167.2; same shape, same noise band).
- **~300 t/s aggregate** across three concurrent 512-token streams
  (102.8 t/s per stream, round 1 and 2 both) — the trial arm's 280.2.
- **2,877 t/s prefill at 106,418 tokens** — the trial arm; the production boot
  has not re-run that depth.
- The 27B record it replaces: 76.6 decode, 109.4 aggregate across two streams,
  691.7 prefill at 166k. Beaten on every axis.
- The **first request after a boot is a warmup artifact** (37.7 t/s class) —
  never report it. And never time a wall-clock aggregate on a generation that
  stops on its own: a 24-token countdown measured 72 t/s across three streams
  while the same shape with `ignore_eos` gave 300.

## Measured — the exact-KV boot (floors, not a clean A/B)

Instrument: `/v1/chat/completions` `timings` (this fork refuses the raw
`/completion` path for greedy runs — see *Contract traps*), greedy,
`ignore_eos`, 512 tokens, first request discarded.

|Arm|q8_0 3-slot record|f16 2-slot, live co-tenancy|
|---|---|---|
|single-stream decode|168.7|**134.1 – 141.5** over five back-to-back runs; 145.8 and 163.2 in a lighter minute|
|concurrent aggregate|≈300 across 3 (102.8 each)|**256.6 / 262.4 / 265.1** across 2 (≈128–133 each)|
|prefill at ~106k|2,877 at 106,418|**3,159.5 at 106,282**|
|decode at 106k depth|never measured|**71.5**|

**These are floors, not the shape's rates.** Someone else was driving the
endpoint through the whole window — 56 new prompts on the other slot in three
minutes, `nvidia-smi` at 96 % GPU-util — which is the exact condition the
Benchmarking Policy forbids, and a contended deep arm even came back truncated
(a 512 KB body accepted as `prompt_n = 4`). What the floors already support:
**per-conversation concurrency improved** (two streams hold ~130 t/s each where
three held 102.8), endpoint aggregate sits ~12 % under the 3-slot record,
prefill beat the record (+9.8 %) while contended, and single-stream decode is
between parity and −20 % against it. The 1.882× KV byte bill is real; its true
size on this card is still unmeasured, and quoting 134 t/s or 163 t/s as "the"
number would be a guess in either direction. Re-run the same arms with the box
quiet before the figure is carried anywhere else.

## Contract traps

- **`--metrics` is mandatory.** Without it llama.cpp answers **501** on
  `/metrics`, so the Prometheus scrape reads DOWN while `/health` answers 200 —
  that is how this endpoint sat red across four payload generations

  (`up{job="frank-zealot"}` oldest retained sample, 2026-10-11, is already 0).
- **Bare `/completion` is a dead instrument on this fork build.** A greedy
  `ignore_eos` run answers **500 — "The model produced output that does not
  match the expected Content-only format"**, because the fork guards the raw
  path the moment reasoning markup leaks (older builds only logged `failed to
  parse reasoning content` and answered anyway). The honest per-request
  instrument on THIS build is `/v1/chat/completions`, whose non-stream response
  carries the same `timings` object (`prompt_n`, `prompt_per_second`,
  `predicted_n`, `predicted_per_second`). `AGENTS.md` still names the
  `/completion` `timings` object as this endpoint's instrument — on build 10777
  read it off the chat path.
- **Q8_0 mmproj, not BF16.** The publisher's card: the BF16 tower "can crash
  llama-server when several image requests run at once; Q8_0 is not affected" —
  and `--parallel 2` is still precisely that shape.
- **Only the `image_url` content part reaches the model.** The legacy top-level
  `images` field bumps `usage.prompt_tokens` and the model still invents an
  answer. Probe with a solid color and expect the exact word.
- **≥256 px probe images** or the model answers "video".
- **Thinking tokens count against `max_tokens`.** A vision probe at 64 tokens
  returns EMPTY `content` + `finish_reason: "length"` — the endpoint looks blind
  while it is only mid-thought. Budget ≥512.
- **`presence_penalty` has no GGUF metadata key**, so the one recipe value that
  must reach the argv is this one. The serve script uses
  `${PRESENCE_PENALTY=1.0}` — **not `:=`** — because `""` is the documented
  opt-out and `:=` would silently re-add the flag (caught by the seam, fixed
  2026-10-18).
- **Spec flags move as a SET.** A bare `--model-draft`/`--spec-type` arm without
  the whole set has segfaulted at load on this fork family in the past;
  `SPEC_TYPE=none` strips all of them.
- **Base alignment is the censored base lineage, and the CRACK swap is NOT
  one line here.** `/home/dank/models/Bonsai2-CRACK/` matches the **27B**
  geometry, not this 9B file; a dealigned 9B would have to be fetched.
- **A raw `/completion` request** logs `failed to parse reasoning content
  (… parse_reasoning=true)` when the model closes a bare prompt with `` —
  cosmetic (`probe_request.cpp:789`); `/v1/chat/completions` parses
  `reasoning_content` correctly. Do not lower `VERBOSITY`: the KV/buffer ledger
  is what VRAM forensics read.
- **A failed stop leaves a `failed` unit, and an orphan keeps answering.** On
  2026-10-18 the trial stop hit `State 'stop-sigterm' timed out. Killing.` →
  `Active: failed (Result: timeout)`, while the hand-booted arm kept serving
  7771 — a live port with no unit behind it. `TimeoutStopSec=180` is now on the
  unit (llama.cpp needs the drain time to release slots and the card); the
  hand-booted arm must be killed by PID, never `pkill -f llama-server`.
- **A leftover arm drop-in hijacks the production boot, and no health check
  sees it.** The sanctioned bench method is a transient
  `Environment=FRANK_ZEALOT_ENV_FILE=/tmp/<arm>.env` drop-in, and an arm env
  `source`s the repo env and then overrides it. On 2026-10-18 a stale
  `frank-zealot.service.d/ptq1.conf` — written 03:29 for the 27B **PTQ1_0** arm
  Frank had ordered at 02:36 and measured from 03:14 (deploy ledger below) —
  was still installed, so the cutover deploy restarted
  the unit and booted THAT: `--model …/Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf
  --ctx-size 401408 --parallel 3 --batch-size 3072 --ubatch-size 1536`, three
  slots of 133,888, 21,188 MiB used. `/health` was 200, `/v1/models` said
  `frank-zealot` (the alias is a flag, not a fact), and the Playwright gate
  passed 33/33 — none of those read the payload. **After deploying an endpoint
  that was recently benched: `systemctl show -p DropInPaths`, then
  `tr '\0' ' ' </proc/$pid/cmdline`, then the ledger — in that order.** Retire
  the drop-in to `~/.AGENTTRASH/<date>-<topic>/`, `daemon-reload`, restart.

## Compaction hub

`models.yml` routes `compactionModel` for `frank-archon`, `frank-dragoon` and
`frank-zealot` here — 262,144 is the widest window in the fleet and the only one
that comfortably fits a worst-case compaction input (~170k tokens; the clamp is
`min(maxTokens 16384, contextWindow − est − 4096)`, compaction fires at 85 % of
the window ≈ 222,822 est, worst case ≈ 245,891 ≤ 262,144). What the reslice to
**two** slots does cost is here: a compaction request still runs beside ONE live
generation, but with two conversations already busy a third waits — the Gemma4
era's `--parallel 1` serialization law at half strength. The window itself is
untouched, and the window is the term the compaction contract depends on.

## Deploy ledger (2026-10-18)

- `./scripts/deploy.sh dankclaw --service frank-zealot` at 13:16 PDT; the e2e
  gate ran 33/33. The first boot was the hijacked arm above; the drop-in went
  to `~/.AGENTTRASH/20261018-zealot-ptq1-arm/` and the 13:19 restart is the
  ledger recorded here.
- Live probes on the shipped boot: chat `27*59` → **1593**; `where_am_i` tool
  call fired with schema-correct empty arguments; 512 px solid-red PNG →
  **Red** at 278 prompt tokens with `process_mtmd` chunk encoding in the
  journal; `/props` reports `n_ctx 262144` and `presence_penalty 1.0`;
  `/metrics` answers 200 with 15 `llamacpp:*` families and
  `up{job="frank-zealot"}` is **1** in Prometheus for the first time in its
  retained history.
- OpenWebUI picker rows re-keyed in the live DB (`Connection.backup()` snapshot
  `webui.db.bak-20261018` taken first; `frank-dragoon`'s `context.length` also
  corrected 262,144 → 200,704 — see [[OpenWebUI]]).
- **What the bench window left behind is not garbage — the PTQ1_0 arm was
  ordered, measured, and then scratched by Frank himself.** The instruction at
  **02:36** was: run frank-zealot on `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, the
  **Q8** mmproj, batch/ubatch **3072/1536**, and as much context as fits in
  **two** contexts. The unit booted it at **03:14** (journal:
  `type ptq1_0: 402 tensors`, `file type = PTQ1_0 - 1.75 bpw ternary`, mmproj
  loaded, 2 × 200,704) and served 24 minutes of traffic. Its own
  `print_timing` lines measure **~48 t/s** single-stream decode
  (48.0 / 47.9 / 47.6 / 49.0 t/s on 512 / 229 / 512 / 451-token generations),
  25–27 t/s per stream with two streams live, and prefill **1,295 t/s at
  30,637 → 943 at 97,856 → 801 t/s at 141,365** tokens. It bought the
  1,201 MiB the fetch script promised and paid for it on decode, exactly as the
  publisher's Ampere row warned (the PQ2_0 record for the same weights is
  76.6 t/s). At **03:33** Frank scratched the whole 27B plan and ordered the
  9B PQ2_0 + MTP shape deployed here; the arm process ended at 03:38.
- The actual failure was **documentation, not measurement.** An approved
  payload that survives only as a systemd drop-in plus a journal window is,
  a day later, indistinguishable from a rogue arm — which is why the 13:16 boot
  read as a hijack. Payload decisions belong in this runbook the moment they
  are made. The evidence is still perishable: `/tmp/zealot-arm-current.env` and
  the 9B `/tmp/arm-*.jsonl` results die on reboot, and only the journal
  (`journalctl -u frank-zealot --since "2026-10-18 03:10"`) holds the PTQ1_0
  numbers. `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + hardlinked
  mmproj) stays on disk as the 27B rollback.

## Fallback rungs

Cheapest first; every one is a single line of `scripts/frank-zealot.env`, and
the KV figures are for the 524,288-position pool unless stated otherwise:

1. `UBATCH_SIZE=1024` — hands back **1,306 MiB** of compute buffers measured on
   this card (prefill −17 %).
2. `CACHE_TYPE_V=q8_0` — exact K, quantized V: the pool goes 16,384 →
   **12,544 MiB**, freeing 3,840, and it is the halfway door back.
   `--flash-attn on` is what keeps quantized V legal.
3. `CTX_SIZE=401408` — two 200,704 conversations: exact KV at
   **12,544.00 MiB**, same 3,840 freed, but `models.yml` `contextWindow` MUST
   follow the per-slot number or long turns start 400-ing.
4. Revert to `q8_0` K + `q8_0` V at `CTX_SIZE=786432 PARALLEL=3` — the shape
   this one replaced: 13,056.00 MiB of cache and the THIRD conversation back.
5. `CACHE_TYPE_K/V=q4_0` — the far rung: 10,240 B/token ⇒ **5,120 MiB** here
   (the 27B era measured q4_0 at 16,392 used / 7,733 free at 2 × 200,704).

Any retune has to keep the seam's fit gate green — it is the only instrument
that can see a boot that would die at `cudaMalloc`.

## Rollback

Revert the swap commit. On disk: the **27B payload**
(`/home/dank/models/Bonsai2-PQ2_0/`) + the **10754** fork tree, the Qwen3.5-9B
Q6_K payload (`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`) + master CUDA build,
the Gemma4, MiMo, Swift and Ornith payloads. Then
`./scripts/deploy.sh dankclaw --service frank-zealot`.

## Related

- [[frank-zealot-llama-prism-bonsai2-pq20]] — the retired 27B era (rollback target)
- [[Llama Cpp Serving Sizing]] — the two-boot VRAM pricing method
- [[frank-dragoon-flash-server-v2-native]] — the other 2026-10-18 cutover
- [[deployment]] — deploy flow and incidents