scripts/frank-zealot.env

@@ -32,9 +32,12 @@ PARALLEL=3
 # on [[frank-zealot]]); 1024×6 keeps it a clean multiple of the ubatch.
 BATCH_SIZE=6144
 # Ubatch is a prefill knob (decode never moves with it) and it sets the
-# compute-buffer size — which is what decides whether a drafter fits: 1024
-# hands back ~1,306 MiB beside the draft context, and it is the rung the
-# measured MTP arm ran (213.5 t/s at 1024 against 109.2 at 2048).
+# compute-buffer size, which decides whether a drafter fits. 1024 is the rung
+# every measured MTP arm here ran on (213.5 t/s against 109.2 at 2048). Its
+# VRAM credit is SHAPE-specific: the 786,432-pool arm census showed ~1,306 MiB
+# back, this boot measured 2,089.88 MiB of target compute against 2,496.13 at
+# ubatch 2048 — 406 MiB back, with the split moving (CUDA0 +257, CUDA_Host
+# −663). The drafter is funded by CACHE_TYPE_V below, not by this knob.
 UBATCH_SIZE=1024
 # K exact, V in the 8-bit class: 16,384 + 8,704 = 25,088 B/token (8,192
 # elements per token per cache × 2 B for f16, × 1.0625 for q8_0 — there is no

scripts/serve-frank-zealot.sh

@@ -37,10 +37,12 @@
 # accepts well (0.798/0.617 per position, mean length 2.41, measured 2026-10-18);
 # what made it lose that evening was the bill, not the drafts — the draft context
 # is built against the target's WHOLE pool, so blk.32 adds 4,096 B/position
-# (2,352.00 MiB at 602,112) plus ~0.45 GiB of buffers at this ubatch, and the
-# recurrent state triples. Dropping V to q8_0 (−3,840 MiB at this pool) and the
-# ubatch to 1024 (−~1,306 MiB of compute buffers) is how those two bills got
-# paid; the arm census and the rates are on [[frank-zealot]]. SPEC_TYPE=none is
+# (2,352.00 MiB at 602,112) plus 480.97 MiB of its own compute buffers on the
+# card, and it triples the recurrent state per slot (452.25 MiB for three cells
+# against 50.25 a cell with the drafter off). CACHE_TYPE_V=q8_0 is what pays
+# that: 7,680 B/token cheaper than exact V, 4,410 MiB at this pool. The ubatch
+# drop to 1024 contributes ~406 MiB here, NOT the ~1,306 the 786,432-pool census
+# showed — that figure does not transfer between shapes. SPEC_TYPE=none is
 # the rollback and strips the whole flag set.
 #
 # Sampling: the checkpoint carries the publisher's thinking recipe as GGUF
@@ -81,8 +83,8 @@ fi
 # Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
 # The logical batch is VRAM- and throughput-neutral at a fixed ubatch, so 6144
 # (1024×6) is free; the ubatch is what sets the compute-buffer size, and 1024 is
-# the rung that leaves room for the draft context (−~1,306 MiB against 2048, and
-# the measured MTP arm ran here: 213.5 t/s at 1024 against 109.2 at 2048).
+# the rung every measured MTP arm here ran on (213.5 t/s at 1024 against 109.2
+# at 2048) — worth ~406 MiB at THIS shape, not the ~1,306 the old pool showed.
 : "${BATCH_SIZE:=6144}"
 : "${UBATCH_SIZE:=1024}"
 # Asymmetric KV: f16 K (the engine's own default, spelled out so the contract is

serverAppConfigs-docs/LLM/frank-zealot.md

@@ -47,11 +47,12 @@ buys both and costs the keys their precision.
 **MTP is ON (Frank, 2026-10-19).** The drafter was always good — 2026-10-18 measured acceptance 0.798 / 0.617 per
 position, mean accepted length 2.41, and the publisher's card says depth 2 works best — and arming it is a
 budget decision, not a flag: the draft context is built against the *target's* pool, so `blk.32` bills 4,096
-B/position (2,352.00 MiB at 602,112) plus ~0.45 GiB of buffers at this ubatch and its own recurrent cells.
-`CACHE_TYPE_V=q8_0` (−3,840 MiB against exact V at this pool) and `UBATCH_SIZE=1024` (−~1,306 MiB of compute
-buffers, and the rung every measured MTP arm here ran on) are what pay for it; `UBATCH_SIZE=2048` beside an armed
-drafter is the shape that halved decode (167.2 → 109.2 t/s). `--spec-draft-type-k`/`--spec-draft-type-v` (default
-f16, verified in this build's `--help`) would cut the draft's bill again and stay at their default; there is no
+B/position (2,352.00 MiB at 602,112) plus 480.97 MiB of its own CUDA0 buffers, and it triples the recurrent
+state (452.25 MiB for three cells against 150.75 with the drafter off). `CACHE_TYPE_V=q8_0` is what pays that:
+7,680 B/token cheaper than exact V = **4,410.00 MiB** at this pool. `UBATCH_SIZE=1024` — the rung every measured
+MTP arm here ran on — contributes only ~406 MiB at THIS shape, not the ~1,306 the 786,432-pool census showed,
+and `UBATCH_SIZE=2048` beside an armed drafter is still the shape that halved decode (167.2 → 109.2 t/s).
+`--spec-draft-type-k`/`--spec-draft-type-v` (default f16, verified in this build's `--help`) would cut the draft's bill again and stay at their default; there is no
 `--spec-draft-model` — the draft rides the main model's weights, so a draft file would load the trunk twice.
 `SPEC_TYPE=none` is the rollback and strips the whole set.
 
@@ -62,23 +63,47 @@ longer prompt with a 400, never a truncation — plus `maxTokens: 16384`, `input
 only `xhigh|medium|low` and raises on anything else), `extraBody` 1.0 / 0.95 / 20 / `min_p 0`, and the
 `compactionModel` of [[frank-archon]], [[frank-dragoon]] **and** frank-zealot all pointing here.
 
-**This shape ships unbenchmarked, by law.** Frank ordered the reslice, not a
-number, so the evidence a swap needs is the seam's fit gate (it prices cache +
-draft cache + recurrent states against the 24,123 MiB the driver leaves
-addressable, and refuses the two boots this shape can walk into: exact V with the
-drafter armed, and the retired dtype pair at the retired pool), the deployer's
-health gate, and the first verbosity-4 ledger of the new dtype pair — which is
-recorded on this page because projections drift, and nowhere else. The previous
-era's exact-KV boot and its rate floors live in that era's row below; those floors
-were measured while another consumer drove the other slot (56 prompts in three
-minutes, 96 % GPU-util), the exact condition a bench window exists to exclude.
+**No rates, by law.** Frank ordered the reslice, not a number: the evidence a swap needs is
+the seam's fit gate, the deployer's health gate, and the boot ledger below. The previous era's
+exact-KV boot and its rate floors live in that era's row; those floors were measured while
+another consumer drove the other slot (56 prompts in three minutes, 96 % GPU-util), the exact
+condition a bench window exists to exclude.
+
+**The 2026-10-19 boot, measured** — verbosity-4 ledger of `ba69ea5`, started 03:21:56 PDT.
+Recorded here because projections drift, and nowhere else:
+
+| Term | Ledger |
+|---|---|
+| Target KV over the 602,112 pool | **14,406.00 MiB** — K (f16) 9,408.00 + V (q8_0) 4,998.00, `200704 cells, 8 layers, 3/3 seqs` |
+| Draft KV (`blk.32`, its own f16 context) | **2,352.00 MiB** — 1,176.00 + 1,176.00, `200704 cells, 1 layers, 3/3 seqs`: the whole pool, again |
+| Recurrent state | **452.25 MiB** — `3 cells, 32 layers, 3 seqs, 2 rs_seq`, R (f32) 20.25 + S (f32) 432.00 |
+| Compute buffers | target 1,665.00 CUDA0 + 424.88 CUDA_Host; draft context 480.97 + 424.86 |
+| Weights | 2,283.29 CUDA0 + 257.66 CPU-mapped |
+| Projector | 595.29 model + 248.10 compute + 24.93 CPU (its 868.32 worst case is still charged AFTER the engine's fit check) |
+| Engine's own verdict | projected 21,639 vs 23,404 free, then **"cannot meet free memory target of 1,867 MiB, need to reduce device memory by 102 MiB"** |
+| Settled | **22,870 used / 1,255 free**, `n_slots = 3`, `n_ctx_slot = 200704`, `kv_unified = false` |
+
+This is the first boot on this card that does **not** meet llama.cpp's own free-memory target,
+and 1,255 MiB settled sits under the ~2 GiB band every previous era held. It boots, `/health`
+answers, and the seam's gate still passes — it priced 22,428 MiB, conservative by 442 against
+this ledger. The cheapest rungs back into the band are `CTX_SIZE=401408` (two conversations) or
+`SPEC_TYPE=none`; both are one line and neither changes the payload.
+
+**The drafter is engaged, measured on the same boot** (three ordinary requests, not a bench):
+acceptance 0.706 and 0.763 accepted-per-generated, mean accepted length 2.41 / 2.53,
+per-position rates (0.882, 0.765) and (0.871, 0.699) — depth 2 earning its keep at 0.3–1.5 MiB
+of draft state per saved prompt. The three probes that prove a swap: chat → `"zealot alive"`
+with 448 completion tokens (at `max_tokens: 64` the same request returns EMPTY content with
+`finish_reason: length` — the thinking-budget trap, not a fault); tools →
+`finish_reason: tool_calls` with `{"a":24,"b":7}`; vision → a 512 px solid-red PNG answers
+"Red" at `prompt_tokens` 280.
 
 **Fallback rungs**, cheapest first — one line of `frank-zealot.env` each, and each must keep the seam's fit gate green:
 
 | Knob | Effect from the shipped shape |
 |---|---|
-| `SPEC_TYPE=none` | the drafter off: hands back 2,352 MiB of draft cache plus its buffers and recurrent cells, at the cost of the ~+28 % decode the 2026-10-18 census measured whenever the card has room |
-| `UBATCH_SIZE=2048` | +~1,306 MiB of compute buffers and +4.6 % prefill — and the rung that HALVED decode beside an armed drafter (167.2 → 109.2 t/s) |
+| `SPEC_TYPE=none` | the drafter off: hands back its 2,352.00 MiB of cache, 480.97 MiB of CUDA0 buffers and the 301.50 MiB of rollback planes it adds to the recurrent state (~3.1 GiB), at the cost of the ~+28 % decode the 2026-10-18 census measured whenever the card has room |
+| `UBATCH_SIZE=2048` | +4.6 % prefill and only ~406 MiB of compute buffers at THIS shape (the ~1,306 MiB the 786,432-pool census showed does not transfer) — and the rung that HALVED decode beside an armed drafter (167.2 → 109.2 t/s) |
 | `CTX_SIZE=401408` | two 200,704 conversations: frees 4,802 MiB of cache — `contextWindow` MUST follow the per-slot number |
 | `CACHE_TYPE_V=f16` | exact V: +4,410 MiB at this pool with the drafter armed, which is a `cudaMalloc` death the seam refuses — legal only once the drafter or a slot is gone |
 | `CACHE_TYPE_K=q8_0` | the fully quantized pair (17,408 B/token): frees 4,410 MiB and moves the quantization error into the keys |
@@ -127,6 +152,10 @@ minutes, 96 % GPU-util), the exact condition a bench window exists to exclude.
   `tests/frank_zealot_unit.bats`'s fit gate can see a boot that would die at `cudaMalloc`, so every retune keeps it
   green — and the first verbosity-4 ledger plus settled `nvidia-smi` of a new dtype belongs on this page, because
   projections drift.
+  The same trap one knob down: the ubatch 2048 → 1024 step handed back **~1,306 MiB** at the 786,432 / q8_0-pair
+  arm and **406 MiB** at the 602,112 / f16-K + q8_0-V boot (target compute 2,496.13 → 2,089.88, with CUDA0 going
+  UP 257 while CUDA_Host went down 663). Changing the dtype or the pool moves the compute buffers too, so a
+  credit measured at one shape is not a budget line at the next.
   The two boots that priced non-KV: 2 × 262,144 **without** the projector — 20,190 used, KV 12,800 ⇒ non-KV
   **7,390 MiB**; 3 × 196,608 **with** it — 22,972 used, KV 14,113 ⇒ non-KV **8,859 MiB**. The difference prices the
   vision tower at **~1.4 GiB for a 918 MB file**, because the encoder's activation and compute buffers ride with it.