serverAppConfigs/serverAppConfigs-docs/LLM/frank-zealot.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 3 × 200704, f16 K + q8_0 V, MTP armed	The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws.	llm frank-zealot dankclaw llama-cpp gguf rtx3090 endpoint	2026-10-18

frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0-MTP, 3 × 200,704, f16 K + q8_0 V, MTP armed

Dankclaw's RTX 3090 endpoint — http://192.168.0.143:7771, wire id frank-zealot, the fleet's compaction hub (three conversations at 200,704 each, the window the other two endpoints also cap at). One page endpoint: eras are rows below, not pages. Siblings: frank-archon, frank-dragoon. Deploy with ./scripts/deploy.sh dankclaw --service frank-zealot; swap mechanics in Model Swap and Rollback, sizing math in Llama Cpp Serving Sizing. scripts/frank-zealot.env and scripts/serve-frank-zealot.sh are the authority for the shape; where any prose disagrees with them, the prose is the bug.

Current shape

Item	Value
Checkpoint	/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf — 2,675,410,720 B, LFS sha256 c79fcdfaf6b7bf47…
Projector	…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf — 624,229,664 B, sha256 e0cb6f6988cea323… — Q8_0 is mandatory, not a preference (see Contract)
Fetch	scripts/download-frank-zealot-bonsai9.sh [DEST] (--verify-only; -f, byte count, LFS sha256 per file)
Runtime	/home/dank/GitPackages/llama-prism-b10777/llama-server — PrismML fork, 0.2.0-dev (build 10777, commit e3de71522); the only loader for this payload
Trio	scripts/frank-zealot.service (root, relinked, User=dank, Restart=on-failure, RestartSec=10, LimitMEMLOCK=infinity, TimeoutStopSec=180) · scripts/frank-zealot.env · scripts/serve-frank-zealot.sh
Address / alias	BIND_ADDR=0.0.0.0, LISTEN_PORT=7771, ALIAS=frank-zealot (a flag, not a fact)
Flags	--ctx-size 602112 --parallel 3 ⇒ 200,704 per slot · --cache-type-k f16 --cache-type-v q8_0 (asymmetric, 14,406.00 MiB) · --flash-attn on · --batch-size 6144 --ubatch-size 1024 · --n-gpu-layers 99 · --jinja --reasoning auto --reasoning-format deepseek · --metrics · --mmproj · --log-verbosity 4 · --presence-penalty 1.0 · --spec-type draft-mtp with --spec-draft-n-max 2 --spec-draft-n-min 0 --spec-draft-p-min 0

Why this shape. general.architecture qwen35: 33 blocks (32 trunk + blk.32 = the MTP block) full_attention_interval 4 ⇒ 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, nextn_predict_layers 1 (the head is in the file), trained window 262,144, general.sampling.* 1.0/0.95/20. Only the 8 full layers bill per token and the DeltaNet layers add ~50 MiB of recurrent state per slot, so the shape is a single budget equation at 25,088 B/token (f16 K 16,384 + q8_0 V 8,704): 14,406.00 MiB of cache over a 602,112 pool + 2,352.00 MiB of draft cache (blk.32 at its own f16 rate, billed over the WHOLE pool) + ~7.7 GiB of weights, projector, buffers and states, inside the 24,123 MiB the driver leaves addressable. Exact V (32,768 B/token ⇒ 18,816 MiB here) buys neither the third conversation nor the drafter; the fully quantized pair (17,408 B/token) buys both and costs the keys their precision.

MTP is ON (Frank, 2026-10-19). The drafter was always good — 2026-10-18 measured acceptance 0.798 / 0.617 position, mean accepted length 2.41, and the publisher's card says depth 2 works best — and arming it is a budget decision, not a flag: the draft context is built against the target's pool, so blk.32 bills 4,096 B/position (2,352.00 MiB at 602,112) plus 480.97 MiB of its own CUDA0 buffers, and it triples the recurrent state (452.25 MiB for three cells against 150.75 with the drafter off). CACHE_TYPE_V=q8_0 is what pays that: 7,680 B/token cheaper than exact V = 4,410.00 MiB at this pool. UBATCH_SIZE=1024 — the rung every measured MTP arm here ran on — contributes only ~406 MiB at THIS shape, not the ~1,306 the 786,432-pool census showed, and UBATCH_SIZE=2048 beside an armed drafter is still the shape that halved decode (167.2 → 109.2 t/s). --spec-draft-type-k/--spec-draft-type-v (default f16, verified in this build's --help) would cut the draft's bill again and stay at their default; there is no --spec-draft-model — the draft rides the main model's weights, so a draft file would load the trunk twice. SPEC_TYPE=none is the rollback and strips the whole set.

Harness contract (~/.omp/agent/models.yml, provider llamacpp (dankclaw-3090), id frank-zealot): contextWindow: 200704 — the per-slot window, because llama.cpp divides --ctx-size across slots and answers a longer prompt with a 400, never a truncation — plus maxTokens: 16384, input: [text, image] compat.reasoningContentField: reasoning_content, compat.reasoningEffortMap (load-bearing: the template takes only xhigh|medium|low and raises on anything else), extraBody 1.0 / 0.95 / 20 / min_p 0, and the compactionModel of frank-archon, frank-dragoon and frank-zealot all pointing here.

No rates, by law. Frank ordered the reslice, not a number: the evidence a swap needs is the seam's fit gate, the deployer's health gate, and the boot ledger below. The previous era's exact-KV boot and its rate floors live in that era's row; those floors were measured while another consumer drove the other slot (56 prompts in three minutes, 96 % GPU-util), the exact condition a bench window exists to exclude.

The 2026-10-19 boot, measured — verbosity-4 ledger of ba69ea5, started 03:21:56 PDT. Recorded here because projections drift, and nowhere else:

Term	Ledger
Target KV over the 602,112 pool	14,406.00 MiB — K (f16) 9,408.00 + V (q8_0) 4,998.00, 200704 cells, 8 layers, 3/3 seqs
Draft KV (blk.32, its own f16 context)	2,352.00 MiB — 1,176.00 + 1,176.00, 200704 cells, 1 layers, 3/3 seqs: the whole pool, again
Recurrent state	452.25 MiB — 3 cells, 32 layers, 3 seqs, 2 rs_seq, R (f32) 20.25 + S (f32) 432.00
Compute buffers	target 1,665.00 CUDA0 + 424.88 CUDA_Host; draft context 480.97 + 424.86
Weights	2,283.29 CUDA0 + 257.66 CPU-mapped
Projector	595.29 model + 248.10 compute + 24.93 CPU (its 868.32 worst case is still charged AFTER the engine's fit check)
Engine's own verdict	projected 21,639 vs 23,404 free, then "cannot meet free memory target of 1,867 MiB, need to reduce device memory by 102 MiB"
Settled	22,870 used / 1,255 free, n_slots = 3, n_ctx_slot = 200704, kv_unified = false

This is the first boot on this card that does not meet llama.cpp's own free-memory target, and 1,255 MiB settled sits under the ~2 GiB band every previous era held. It boots, /health answers, and the seam's gate still passes — it priced 22,428 MiB, conservative by 442 against this ledger. The cheapest rungs back into the band are CTX_SIZE=401408 (two conversations) or SPEC_TYPE=none; both are one line and neither changes the payload.

The drafter is engaged, measured on the same boot (three ordinary requests, not a bench): acceptance 0.706 and 0.763 accepted-per-generated, mean accepted length 2.41 / 2.53, per-position rates (0.882, 0.765) and (0.871, 0.699) — depth 2 earning its keep at 0.3–1.5 MiB of draft state per saved prompt. The three probes that prove a swap: chat → "zealot alive" with 448 completion tokens (at max_tokens: 64 the same request returns EMPTY content with finish_reason: length — the thinking-budget trap, not a fault); tools → finish_reason: tool_calls with {"a":24,"b":7}; vision → a 512 px solid-red PNG answers "Red" at prompt_tokens 280.

Fallback rungs, cheapest first — one line of frank-zealot.env each, and each must keep the seam's fit gate green:

Knob	Effect from the shipped shape
SPEC_TYPE=none	the drafter off: hands back its 2,352.00 MiB of cache, 480.97 MiB of CUDA0 buffers and the 301.50 MiB of rollback planes it adds to the recurrent state (~3.1 GiB), at the cost of the ~+28 % decode the 2026-10-18 census measured whenever the card has room
UBATCH_SIZE=2048	—
CTX_SIZE=401408	two 200,704 conversations: frees 4,802 MiB of cache — contextWindow MUST follow the per-slot number
CACHE_TYPE_V=f16	exact V: +4,410 MiB at this pool with the drafter armed, which is a cudaMalloc death the seam refuses — legal only once the drafter or a slot is gone
CACHE_TYPE_K=q8_0	the fully quantized pair (17,408 B/token): frees 4,410 MiB and moves the quantization error into the keys

Eras

Window	Payload	Runtime	Shape	Replaced because	Rollback
2026-10-18 21:06 → 2026-10-19	the same 9B MTP file and projector	PrismML fork 10777	—	Frank ordered the third conversation back AND the drafter armed, and exact V funds neither: it bills 18,816 MiB at the pool that pays for the third slot, leaving nothing for the draft's 2,352	one line each — CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none — and redeploy; no payload move
2026-10-18 13:19 → same evening (the boot 68afc6d replaced)	the same 9B MTP file and projector	PrismML fork 10777	—	Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable	one env line — CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0 — and redeploy; no payload move
2026-10-18 03:14 → 13:19 (arm under the unit, then the leftover drop-in)	Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf 5,946,648,928 B + hardlinked Q8 mmproj	PrismML fork 10777	arm: 2 × 200,704, batch/ubatch 3072/1536. The leftover frank-zealot.service.d/ptq1.conf then made the 13:16 cutover deploy boot THIS payload at 3 × 133,888 (new slot, n_ctx = 133888 x3, journal read 2026-10-19) until the 13:19 boot	—	/home/dank/models/Bonsai2-PTQ1_0/ + scripts/download-frank-zealot-bonsai2-ptq1.sh
2026-10-17 → 10-18	Ternary-Bonsai-2-27B PQ2_0 7,206,168,928 B 3907dc16… + Q8_0 mmproj 629,246,976 B	PrismML fork 10754 (~/GitPackages/llama-prism/)	—	the 9B bills half the KV per token (8 full layers, not 16), which funded three slots at the native window and beat it on every rate	/home/dank/models/Bonsai2-PQ2_0/ + the 10754 fork tree, scripts/download-frank-zealot-bonsai2.sh, revert the swap commit
2026-10-07 → 10-17	Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K 7,359,259,008 B c0ba7beb… + BF16 mmproj 921,704,448 B	llama.cpp master 5fc4f3c CUDA sm_86 (~/GitPackages/llama-master/build-cuda/)	2 × 200,704, exact f16 K+V, MTP off (no nextn tensors)	the ternary 27B bought a larger model class at the same window	/home/dank/models/Qwen35-9B-HauhauCS-Q6K/ + the master build, scripts/download-frank-zealot.sh
2026-10-07 (same day)	Gemma4-26B-A4B-QAT Q4_K_M 16,796,015,520 B + BF16 mmproj 1,194,827,776 B + unused gemma4-assistant MTP 251,937,728 B	llama.cpp master CUDA	—	ONE slot serialized compaction; the 9B's lighter weights bought two slots and exact KV	/home/dank/models/Gemma4-26B-A4B/ (ungated — plain curl -fL)
2026-10-03 → 10-05	MiMo-V2.6-Distill-Qwen-9B Q6_K 7,793,710,624 B ef96d05a… + f16 mmproj 918,166,048 B	llama.cpp master CUDA	—	Frank ordered the uncensored 27B with MTP drafted	/home/dank/models/MiMo-Q6K/
2026-10-05 (committed 77decb5→380aafa, never deployed)	Ornith-1.5-9B Q8_0	llama.cpp master CUDA	1 × 262,144, f16 K+V	Swift cut over the same evening while this deploy sat blocked at the whole-tree docs gate	/home/dank/models/Ornith-Q8/; revert to 380aafa's tree
2026-10-05 → 10-07	Swift-1.5-27B-Uncensored RCO-IQ3_XXS-mtp 10,442,827,392 B e13d3e74… + publisher mmproj-BF16.gguf 931,145,888 B	llama.cpp master CUDA	—	the 9B Q6_K rung returned the 200k window and exact KV	/home/dank/models/Swift15-UncIQ3XXS/; the IQ3_S-mtp rung at /home/dank/models/Swift15-IQ3S/; commits da36fa3 → f2575a1 → d8da7a9
2026-10-02 → 10-03	MiMo Q5_K_M 6,876,124,704 B 04cf46f2… + the same f16 mmproj (hardlinked)	llama.cpp master CUDA	—	the ~3.5 GiB it left free bought the Q6_K rung	/home/dank/models/MiMo-Q5KM/
2026-10-01 → 10-02	MiMo Q6_K (same file as above)	llama.cpp master CUDA	2 × 262,144, f16 K + q8_0 V	one night: the Q5 rung bought the 2–3 GiB headroom band back	/home/dank/models/MiMo-Q6K/ (same dir)
2026-10-01 (hours)	MiMo-V2.6 INT8-W8A16-AutoRound, 8 shards 12,891,718,440 B	SGLang in venv ~/GitPackages/sglang-cu128 (torch cu130 + an in-venv CUDA 13.0 nvcc)	—	llama.cpp was the only runtime exposing asymmetric per-tensor KV	payload purged — re-fetch old/download-frank-zealot-sglang-int8.sh; the venv is still on disk
≤ 2026-09 — 6 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete	—	—	—	—	—

Laws learned here

• Addressable ≠ advertised. The card is 24,576 MiB but leaves 24,123 MiB: quote nvidia-smi's free field, never total − used — the driver's reservation is the difference.
• Exact KV and a 786,432 pool cannot coexist here: 32,768 B/token ⇒ 24,576 MiB before a single weight loads (the retired "26,112 MiB" note doubled q8_0's scale overhead); the 27B analogue died at 25,088 MiB with cudaMalloc failed: out of memory. Halving the pool funds exact KV; the per-slot window stops at the trained 262,144 (no YaRN here), so a third conversation is a KV-dtype decision, never more context. The wall's own figures: cudaMalloc failed on a 25,088.00 MiB KV request when the engine's fit projection asked for 33,198 MiB against 23,558 MiB free. It booted f16 at all because the arm env's invented KV_CACHE_TYPE was swallowed by the serve script's :=f16 default — so a failed window became the measurement.
• Exact KV reads 1.882× the bytes per generated token, not 2× (32,768 vs 17,408 B/token, 2026-10-18): a decode drop of that order is EXPECTED and must be measured, while prefill should hold — and this service can afford the dtype precisely because its load is the compaction hub's ~170k-token prefill, which is weight-bandwidth bound and never re-reads the cache per generated token. The dtype is paid in decode and concurrency.
• Never --fit with parallel > 1 — it maximizes the total pool and shrinks each slot: set CTX_SIZE explicitly and keep it divisible by PARALLEL (the seam enforces it).
• KV rate law. full_layers × 4 KV heads × 256 dim = 8,192 elements per token per cache on this arch (the engine prints n_embd_k_gqa = n_embd_v_gqa = 1024), ×2 B (f16), ×1.0625 (q8_0 — 32 codes plus one 2-byte scale per 32-block), ×0.625 (q4_0): 32,768 / 17,408 / 10,240 B/token on this 8-full-layer 9B, and 34,816 for a 16-full-layer 27B at q8_0. Everything else bills separately: DeltaNet recurrent state per slot (150.75 MiB for 3 cells, and an armed depth-2 drafter triples it to 452.25), compute buffers at graph capture, the mmproj after the fit. On gemma4-class hybrids attention_k_eq_v does not shrink the allocation — the cache reserves the K/V pair twice, 6.125 B/elem at f16 K + q8_0 V.
• --flash-attn on is pinned, not decorative: a quantized V exists only on the FA path and the engine falls back silently without it. llama.cpp has no true fp8 KV dtype — q8_0 is the 8-bit class (1.0625 B/elem, ~8.5 effective bits). GGML_CUDA_FA_ALL_QUANTS is deprecated upstream and inert in both trees on this host (no reference anywhere in ggml/src/ggml-cuda/, read 2026-10-19): master's ggml/cmake/common.cmake warns use GGML_CUDA_FA_QUANTS=all instead, and the successor's default — GGML_CUDA_FA_QUANTS="q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16" (ggml/CMakeLists.txt:208) — already compiles the quantized-KV kernels with the old flag OFF (master's build-cuda cache says FA_ALL_QUANTS:BOOL=OFF and it dispatched q8_0/q4_0 KV regardless). Prove a build's quantized-KV ability from its CMakeCache.txt GGML_CUDA_FA_QUANTS line, never from the retired flag or the build command you remember.
• Prices come from boots, not formulas, and an arm belongs to the payload it was measured on: 3 × 180,224 + q8_0 V fits with Q5_K_M (21,684 used / 2,441 free) and does not fit with Q6_K inside the 2–3 GiB band. The mmproj worst case is printed before fitting (estimated worst-case memory usage of mmproj is 873.10 MiB on the Bonsai-2 27B boot) but never billed by the engine's own fit check: that boot projected 21,439 MiB and settled at 22,664. Only tests/frank_zealot_unit.bats's fit gate can see a boot that would die at cudaMalloc, so every retune keeps it green — and the first verbosity-4 ledger plus settled nvidia-smi of a new dtype belongs on this page, because projections drift. The same trap one knob down: the ubatch 2048 → 1024 step handed back ~1,306 MiB at the 786,432 / q8_0-pair arm and 406 MiB at the 602,112 / f16-K + q8_0-V boot (target compute 2,496.13 → 2,089.88, with CUDA0 going UP 257 while CUDA_Host went down 663). Changing the dtype or the pool moves the compute buffers too, so a credit measured at one shape is not a budget line at the next. The two boots that priced non-KV: 2 × 262,144 without the projector — 20,190 used, KV 12,800 ⇒ non-KV 7,390 MiB; 3 × 196,608 with it — 22,972 used, KV 14,113 ⇒ non-KV 8,859 MiB. The difference prices the vision tower at ~1.4 GiB for a 918 MB file, because the encoder's activation and compute buffers ride with it. The first 3-slot deploy (3 × 196,608) booted healthy and left only 1,153 MiB free — that is how the 2–3 GiB band got violated in practice, and why the seam prices non-KV at the measured 9 GiB and refuses that shape.
• Ubatch is a prefill knob. Decode never moves with it and the logical --batch-size is neutral; prefill climbs monotonically to 2048 on this CUDA card (+4.6 % at 8.9k depth; 710 → 1,188 t/s), a prompt shorter than the ubatch cannot see the knob at all (the control row), and a verdict never crosses backends — the 2× R9700 split peaks at 512. Each rung costs ~0.6–0.75 GiB of compute buffers; 2048 is the ceiling for a 27B-on-3090 shape.
• A quant rung is paid in decode: Q5_K_M → Q6_K is +875 MiB of weights and −10 % rate on this bandwidth-bound card, and the byte-bandwidth law predicts a rung's rate within 1 %.
• Speculation pays only with room on the card. The 2026-10-18 arm census (all q8_0 K+V at the 786,432 pool): 3 × 262,144 spec-OFF ubatch 2048 → 17,876 / 6,249 free, 167.2 t/s; spec-ON ubatch 2048 → 22,706 / 1,419, 109.2 t/s; spec-ON ubatch 1024 → 21,400 / 2,725, 213.5 t/s; 2 × 262,144 spec-ON → 213.2; 1 × 262,144 spec-ON → 214.2. That is +28 % decode whenever the card has room, and the doors back are one window per conversation, or q8_0 KV with ubatch 1024. The draft's own bill at that shape was 3,072.00 MiB of KV plus ~1,758 MiB of buffers.
• Draft KV is built against the target's whole pool (~2.4 GiB measured on the 27B Swift boot, not the ~0.4 GiB a proposal-depth estimate suggests). Spec flags move as a SET — SPEC_TYPE=none strips all of them and half a set has segfaulted at load. Two context-backed drafters cannot share a process, so draft-mtp,ngram-mod is the only legal pair, and on a qwen35 hybrid ngram-mod disables recurrent-state rollback so rejected drafts degrade silently.
• Acceptance alone never justifies speculation: 2026-08-25 measured 26.8–35.2 t/s drafting against 53.7 no-spec.
• A checkpoint declaring *_num_hidden_layers MTP while carrying zero mtp.*/nextn.* tensors boots a 4.24 GiB ghost drafter at accept rate: 0.00 — read the safetensors index, not the config.
• MTP is a property of the file: the plain PQ2_0 sibling could not draft at any flag value; only the -MTP export carries blk.N.nextn.*.
• The format is the runtime lock. PQ2_0 is general.file_type 141 over 202 tensors of custom GGML type 142 plus prism.hadamard.* metadata: stock llama.cpp refuses the type, and a loader ignoring the Hadamard transform would dequant wrong, silently (same lock class as ROCmFPX). Build ≥ b10776 is additionally the floor that pairs MTP with a quantized V cache.
• Check arch support in the source tree: grep -c on the packed binary returns 0 for every arch, including the one it serves.
• Base alignment. Bonsai-2 is the censored base lineage. /home/dank/models/Bonsai2-CRACK/ matches the 27B geometry (a one-line MODEL_FILE swap there, identical tensor census); no dealigned 9B exists, so for this payload the refusal escape hatch is a fetch, not a revert.
• A stale bench drop-in hijacks the production boot and no health check sees it. An Environment=FRANK_ZEALOT_ENV_FILE=/tmp/<arm>.env in frank-zealot.service.d/ booted a 27B PTQ1_0 payload while /health answered 200, /v1/models echoed the alias flag and the e2e gate passed 33/33 — none of them read the payload.
• After any bench window: systemctl show -p DropInPaths → tr '\0' ' ' </proc/$pid/cmdline → ledger, in that order; drop-ins retire to ~/.AGENTTRASH/<date>-<topic>/, and payload decisions land on this page the moment they are made, or an approved arm becomes indistinguishable from a rogue one a day later.
• An arm env must use the serve script's own variable names (CACHE_TYPE_K/CACHE_TYPE_V, BIND_ADDR, NGL): an invented KV_CACHE_TYPE is swallowed by : "${CACHE_TYPE_K:=f16}" and the arm boots f16 — an OOM that reads like a model fault.
• Stop, shebang, GPU node. llama.cpp rides out the default stop timeout (TimeoutStopSec=180 is on the unit for that reason) and a hand-booted arm keeps answering 7771 behind a failed unit — kill it by PID, never pkill -f llama-server.
• serve-frank-zealot.sh is a direct ExecStart, so a missing shebang is a 203/EXEC boot loop at Mem peak: 1.9M with the GPU untouched: head -1 is a deploy check.
• No CUDA GPUs are available ⇒ nvidia-modprobe -u -c=0; KV spilled into unified memory halves decode silently, so re-read nvidia-smi after any shape change.

Contract

• /health → {"status":"ok"}; /props reports n_ctx and presence_penalty; /slots shows occupancy; a fourth concurrent conversation waits for a slot — the slot count is the concurrency contract.
• /metrics answers 200 with 15 llamacpp:* families only with --metrics; without it llama.cpp answers 501, which Prometheus reads as DOWN while /health stays 200. The scrape target is host.containers.internal:7771/metrics and the oldest retained sample of up{job="frank-zealot"} (2026-10-11) is already 0 — this target was never successfully scraped, across every payload era. METRICS=0 is the documented opt-out and re-breaks it; the counters cost nothing on the request path.
• Reasoning arrives in reasoning_content (--reasoning-format deepseek) and counts against max_tokens: a short budget yields empty content with finish_reason: "length", which reads as a broken model, not a mid-thought one. The slot prefix cache is live (cached_tokens in usage).
• A client that sends no sampler is not greedy — the engine applies the payload's own general.sampling.* (1.0 / 0.95 / 20). presence_penalty is the one recipe value with no GGUF metadata key, so it must reach argv; the serve script uses ${PRESENCE_PENALTY=1.0}, deliberately not :=, because "" is the documented opt-out.
• Vision: only the OpenAI image_url content part reaches the model — the legacy top-level images field bumps usage.prompt_tokens while the model invents an answer. Probe with a ≥ 256 px solid color (a postage stamp answers "video") and budget ≥ 512 tokens; a 512 px image bills ~280 prompt tokens.
• Q8_0 mmproj, not BF16: the publisher warns the BF16 tower "can crash llama-server when several image requests run at once", which is exactly the --parallel 3 shape. MMPROJ_FILE= (explicitly empty) is the text-only opt-out and returns ~1.4 GiB; a path at a missing file is a boot refusal, never silent blindness.
• Tools: native via --jinja; finish_reason: tool_calls with valid JSON arguments.
• Two non-faults: a 400 naming 200,704 is one conversation outgrowing its slot, not pool exhaustion; and the raw /completion path is a dead instrument on this fork build — a greedy ignore_eos run answers 500 "The model produced output that does not match the expected Content-only format", because build 10777 guards the raw path the moment reasoning markup leaks (older builds only logged failed to parse reasoning content and answered anyway). The honest per-request instrument here is /v1/chat/completions, whose non-stream response carries the same timings object (prompt_n, prompt_per_second, predicted_n predicted_per_second). Keep VERBOSITY=4 — 3 is the engine default and cannot prove the KV or draft sizes, while levels above 4 flood the journal.
• llama.cpp does not validate the requested model id: it answers and echoes the served ALIAS, so a rename breaks id-keyed consumers (the OpenWebUI model row, its access_grant, openai.api_configs) and nothing server-side. Alias history: frank-small → frank-secondary (2026-10-06) → frank-zealot (7c77326, 2026-10-09); the service and unit took the name 2026-10-10 (55f87fc).
• The first request after a boot is a warmup artifact, never a measurement: on the Bonsai-2 27B boot request #1 gave 2.6 t/s prefill / 37.7 t/s decode and request #2 gave 1,258 t/s prefill / 54.7 t/s per stream (two concurrent, the 109.4 aggregate in the Eras row) — a 3x spread on one shape. Never time a wall-clock aggregate on a generation that stops on its own (use ignore_eos).

Files

• scripts/frank-zealot.env — the authority for the values above and nothing else: as of 2026-10-18 it carries the knobs plus the one-line reason a reader editing that line cannot recover here. The arm table, the reference boot ledger and the KV derivation all live on this page now; the header used to hold them, and two of its rung figures were wrong (CACHE_TYPE_V=q8_0 and CTX_SIZE=401408 both free 3,840 MiB at the 524,288 pool, not 8,192 and not ~6.7 GiB — the Fallback rungs table above is the corrected statement).
• scripts/serve-frank-zealot.sh — the exec line and its rationale comments; honors FRANK_ZEALOT_ENV_FILE for arm boots; exits 1 naming a missing binary, GGUF or projector rather than half-loading (a restart policy would turn an unfinished download into a crash loop that reads like a runtime fault).
• scripts/frank-zealot.service — root unit, relinked into /etc/systemd/system/.
• scripts/download-frank-zealot-bonsai9.sh (active) · …-bonsai2.sh · …-bonsai2-ptq1.sh · scripts/download-frank-zealot.sh (master-era GGUFs) · old/download-frank-zealot-sglang-int8.sh (SGLang).
• tests/frank_zealot_unit.bats — stub-argv seam plus the VRAM fit gate.
• scripts/deploy-remote.sh — deploy_llm_zealot (payload-path gates, MainPID-hardened health);
• shared/prometheus/prometheus.yml, shared/prometheus/rules/llm-fleet.yml shared/grafana/provisioning/dashboards/llm-fleet.json — the scrape, the alert, the dashboard.
• Live state, not repo files: the OpenWebUI picker row (back up webui.db before re-keying) and ~/.omp/agent/models.yml.