serverAppConfigs/serverAppConfigs-docs/LLM/frank-zealot.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 2 × 262144, exact f16 KV	The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env ships it, every era with its rollback path, the deduplicated laws of this card, the wire contract, and the owning files.	llm frank-zealot dankclaw llama-cpp gguf rtx3090	2026-10-18

frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0-MTP, 2 × 262,144, exact f16 KV

Dankclaw's RTX 3090 endpoint — http://192.168.0.143:7771, wire id frank-zealot, the fleet's compaction hub (widest window it has). One page per endpoint: eras are rows below, not pages. Siblings: frank-archon, frank-dragoon. Deploy with ./scripts/deploy.sh dankclaw --service frank-zealot; swap mechanics in Model Swap and Rollback, sizing math in Llama Cpp Serving Sizing. scripts/frank-zealot.env and scripts/serve-frank-zealot.sh are the authority for the shape; where any prose disagrees with them, the prose is the bug.

Current shape

Item	Value
Checkpoint	/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf — 2,675,410,720 B, LFS sha256 c79fcdfaf6b7bf47…
Projector	…/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf — 624,229,664 B, sha256 e0cb6f6988cea323… — Q8_0 is mandatory, not a preference (see Contract)
Fetch	scripts/download-frank-zealot-bonsai9.sh [DEST] (--verify-only; -f, byte count, LFS sha256 per file)
Runtime	/home/dank/GitPackages/llama-prism-b10777/llama-server — PrismML fork, 0.2.0-dev (build 10777, commit e3de71522); the only loader for this payload
Trio	scripts/frank-zealot.service (root, relinked, User=dank, Restart=on-failure, RestartSec=10, LimitMEMLOCK=infinity, TimeoutStopSec=180) · scripts/frank-zealot.env · scripts/serve-frank-zealot.sh
Address / alias	BIND_ADDR=0.0.0.0, LISTEN_PORT=7771, ALIAS=frank-zealot (a flag, not a fact)
Flags	--ctx-size 524288 --parallel 2 ⇒ 262,144 per slot · --cache-type-k f16 --cache-type-v f16 (exact, 16,384 MiB) · --flash-attn on · --batch-size 4096 --ubatch-size 2048 · --n-gpu-layers 99 · --jinja --reasoning auto --reasoning-format deepseek · --metrics · --mmproj · --log-verbosity 4 · --presence-penalty 1.0 · SPEC_TYPE=none

Why this shape. general.architecture qwen35: 33 blocks (32 trunk + blk.32 = the MTP block), full_attention_interval 4 ⇒ 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, nextn_predict_layers 1 (the head is in the file), trained window 262,144, general.sampling.* 1.0/0.95/20. Only the 8 full layers bill per token (32,768 B at exact f16 ⇒ 16,384 MiB over the pool) and the DeltaNet layers add ~50 MiB of recurrent state per slot: exact KV costs the third conversation and nothing else.

MTP is OFF by measurement, not by absence of a head. The drafter is good — 2026-10-18 measured acceptance 0.798 / 0.617 per position, mean accepted length 2.41, and the publisher's card says depth 2 works best — and it still loses here, because the draft context is built against the target's pool: blk.32 bills 4,096 B/position at its default f16 ⇒ 2,048 MiB at 524,288 positions on a trunk that projects 21,234 MiB of the 24,123 MiB addressable (that projection predicts 2,889 MiB free; replace it with the measured pair). Exact KV and a drafter cannot share this card. --spec-draft-type-k/--spec-draft-type-v (default f16, verified in this build's --help) would cut the draft to 1,088 MiB — legal on b10777, never booted beside exact KV. There is no --spec-draft-model: the MTP draft rides the main model's weights. The env carries the recipe (SPEC_DRAFT_N_MAX=2, SPEC_DRAFT_N_MIN=0, SPEC_DRAFT_P_MIN=0); SPEC_TYPE=none never sends it.

Harness contract (~/.omp/agent/models.yml, provider llamacpp (dankclaw-3090), id frank-zealot): contextWindow: 262144 — the per-slot window, because llama.cpp divides --ctx-size across slots and answers a longer prompt with a 400, never a truncation — plus maxTokens: 16384, input: [text, image], compat.reasoningContentField: reasoning_content, compat.reasoningEffortMap (load-bearing: the template takes only xhigh|medium|low and raises on anything else), extraBody 1.0 / 0.95 / 20 / min_p 0, and the compactionModel of frank-archon, frank-dragoon and frank-zealot all pointing here.

The exact-KV boot, measured (2026-10-18 evening, under the unit, verbosity 4). KV 16,384.00 MiB = 32,768 B/token × 524,288 positions, to the MiB — rate derivation and ledger agree with no slop, which is what buries the doc's old "f16 needs 26,112 MiB and cannot boot" figure for good: that was the 27B's 34,816 B/token applied to the 9B. RS 100.50 MiB (2 cells, one 50.25 MiB cell lighter than the q8_0 era), compute 1,408.06 + 1,088.07 CUDA_Host, mmproj worst case 868.32 MiB charged after the engine's own fit check, settled nvidia-smi 21,176 used / 2,949 free of 24,123, n_slots = 2, n_ctx_slot = 262144, DropInPaths empty, no clamp line, no cudaMalloc error — within 24 MiB of the pre-flight projection. That is a desktop margin: the ~2 GiB of slack is what an image storm has to live inside.

Rates on this shape are FLOORS, not the shape's rates — measured the same evening greedy with ignore_eos at 512 tokens, first request discarded, while another consumer drove the other slot (56 new prompts in three minutes, 96 % GPU-util): the exact condition the bench window exists to exclude, and one contended deep arm even came back truncated (a 512 KB body accepted as prompt_n = 4).

Arm	q8_0 3-slot record	f16 2-slot, contended
single-stream decode	168.7	134.1 – 141.5 (145.8 and 163.2 in a lighter minute)
concurrent aggregate	≈300 across 3 (102.8 each)	256.6 / 262.4 / 265.1 across 2 (≈128–133 each)
prefill at ~106k	2,877 at 106,418	3,159.5 at 106,282
decode at 106k depth	never measured	71.5

What the floors already support: per-conversation concurrency improved (two streams hold ~130 t/s each where three held 102.8), endpoint aggregate sits ~12 % under the 3-slot record, prefill beat the record while contended, and single-stream decode is between parity and −20 %. The 1.882× KV byte bill is real; its true size on this card is still unmeasured, so quoting 134 or 163 t/s as "the" number is a guess in either direction — re-run the same arms with the box quiet before a figure is carried anywhere else.

Fallback rungs, cheapest first — one line of frank-zealot.env each, and each must keep the seam's fit gate green:

Knob	Effect
UBATCH_SIZE=1024	hands back 1,306 MiB of compute buffers; prefill −17 %; it is also the arm that ran the drafter at three windows (213.5 t/s)
CACHE_TYPE_V=q8_0	exact K + quantized V: KV 16,384 → 12,544 MiB, freeing 3,840 (llama.cpp has no true fp8 KV dtype — q8_0 is the 8-bit class)
CTX_SIZE=401408	two 200,704 conversations, exact KV at 12,544 MiB (same 3,840 freed) — contextWindow MUST follow the per-slot number
CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0	the era below this one: 13,056 MiB of cache, the third conversation back
CACHE_TYPE_K/V=q4_0	10,240 B/token ⇒ 5,120 MiB (the 27B era measured q4_0 at 16,392 used / 7,733 free at 2 × 200,704)

Eras

Window	Payload	Runtime	Shape	Replaced because	Rollback
2026-10-18 13:19 → same evening (the boot 68afc6d replaced)	the same 9B MTP file and projector	PrismML fork 10777	3 × 262,144 (pool 786,432), q8_0 K+V = 13,056 MiB, settled 17,956 / 6,169 MiB; 168.7 t/s decode (three runs 168.4–168.7), ~300 t/s over three streams (102.8 each), 2,877 t/s prefill at 106,418	Frank ordered unquantized KV, and that pool cannot survive it: 24,576 MiB of f16 cache > 24,123 addressable	one env line — CTX_SIZE=786432 PARALLEL=3 CACHE_TYPE_K/V=q8_0 — and redeploy; no payload move
2026-10-18, 24 min (arm under the unit)	Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf 5,946,648,928 B + hardlinked Q8 mmproj	PrismML fork 10777	2 × 200,704, batch/ubatch 3072/1536	Frank scratched the 27B plan — ~48 t/s decode vs 76.6 for PQ2_0 on the same weights	/home/dank/models/Bonsai2-PTQ1_0/ + scripts/download-frank-zealot-bonsai2-ptq1.sh
2026-10-17 → 10-18	Ternary-Bonsai-2-27B PQ2_0 7,206,168,928 B 3907dc16… + Q8_0 mmproj 629,246,976 B	PrismML fork 10754 (~/GitPackages/llama-prism/)	2 × 200,704 (pool 401,408), q8_0 K+V = 13,328 MiB, settled 22,664 / 1,461; 76.6 t/s decode, 109.4 aggregate over two streams, 691.7 t/s prefill at 166,027; MTP impossible — no head in the file	the 9B bills half the KV per token (8 full layers, not 16), which funded three slots at the native window and beat it on every rate	/home/dank/models/Bonsai2-PQ2_0/ + the 10754 fork tree, scripts/download-frank-zealot-bonsai2.sh, revert the swap commit
2026-10-07 → 10-17	Qwen3.5-9B-Uncensored-HauhauCS-Aggressive Q6_K 7,359,259,008 B c0ba7beb… + BF16 mmproj 921,704,448 B	llama.cpp master 5fc4f3c CUDA sm_86 (~/GitPackages/llama-master/build-cuda/)	2 × 200,704, exact f16 K+V, MTP off (no nextn tensors)	the ternary 27B bought a larger model class at the same window	/home/dank/models/Qwen35-9B-HauhauCS-Q6K/ + the master build, scripts/download-frank-zealot.sh
2026-10-07 (same day)	Gemma4-26B-A4B-QAT Q4_K_M 16,796,015,520 B + BF16 mmproj 1,194,827,776 B + unused gemma4-assistant MTP 251,937,728 B	llama.cpp master CUDA	1 × 200,704, f16 K + q8_0 V, ubatch 1024, MTP off; 21,868 / 2,257 settled, 139.9 t/s decode (the ~4 B-active MoE is at home here); MTP ON would cost ~1.9 GiB and boot at 319 MiB free	ONE slot serialized compaction; the 9B's lighter weights bought two slots and exact KV	/home/dank/models/Gemma4-26B-A4B/ (ungated — plain curl -fL)
2026-10-03 → 10-05	MiMo-V2.6-Distill-Qwen-9B Q6_K 7,793,710,624 B ef96d05a… + f16 mmproj 918,166,048 B	llama.cpp master CUDA	2 × 200,704, f16 K+V, batch 4096 / ubatch 2048; 21,846 / 2,279 settled, 94.44 t/s decode	Frank ordered the uncensored 27B with MTP drafted	/home/dank/models/MiMo-Q6K/
2026-10-05 (committed 77decb5→380aafa, never deployed)	Ornith-1.5-9B Q8_0	llama.cpp master CUDA	1 × 262,144, f16 K+V	Swift cut over the same evening while this deploy sat blocked at the whole-tree docs gate	/home/dank/models/Ornith-Q8/; revert to 380aafa's tree
2026-10-05 → 10-07	Swift-1.5-27B-Uncensored RCO-IQ3_XXS-mtp 10,442,827,392 B e13d3e74… + publisher mmproj-BF16.gguf 931,145,888 B	llama.cpp master CUDA	2 × 114,688, q8_0 K+V, draft-mtp depth 3, ubatch 2048; 23,120 / 1,005 settled (489 MiB at the 2 × 131,072 shape — the tightest fit ever here), 63.3 t/s at the recipe temp, 72.9 at temp 0, 48.0 with the drafter off	the 9B Q6_K rung returned the 200k window and exact KV	/home/dank/models/Swift15-UncIQ3XXS/; the IQ3_S-mtp rung at /home/dank/models/Swift15-IQ3S/; commits da36fa3 → f2575a1 → d8da7a9
2026-10-02 → 10-03	MiMo Q5_K_M 6,876,124,704 B 04cf46f2… + the same f16 mmproj (hardlinked)	llama.cpp master CUDA	2 × 200,704, f16 K+V, batch 2048 / ubatch 1024; 20,530 used / 4,046 apparent (true free 3,571), 104.84 t/s decode	the ~3.5 GiB it left free bought the Q6_K rung	/home/dank/models/MiMo-Q5KM/
2026-10-01 → 10-02	MiMo Q6_K (same file as above)	llama.cpp master CUDA	2 × 262,144, f16 K + q8_0 V	one night: the Q5 rung bought the 2–3 GiB headroom band back	/home/dank/models/MiMo-Q6K/ (same dir)
2026-10-01 (hours)	MiMo-V2.6 INT8-W8A16-AutoRound, 8 shards 12,891,718,440 B	SGLang in venv ~/GitPackages/sglang-cu128 (torch cu130 + an in-venv CUDA 13.0 nvcc)	2 × 262,144 over a pinned 606,208 pool, fp8_e4m3 KV, NEXTN off; 83.9 t/s decode, ~2,050–3,370 t/s eager prefill, ~60 s boot warm	llama.cpp was the only runtime exposing asymmetric per-tensor KV	payload purged — re-fetch old/download-frank-zealot-sglang-int8.sh; the venv is still on disk
2026-09-17 → 10-01	Gemma4-12B-QAT-Uncensored Q4_K_M 6.87 GiB + BF16 mmproj (it carried an audio encoder) + mtp-gemma-4-12B-it.gguf sidecar 0.24 GiB	llama.cpp master 37b53fd CUDA with -DGGML_CUDA_FA_ALL_QUANTS=ON (~/GitPackages/llama.cpp-mainline/build-cuda/)	4 × 262,144, q8_0 K+V, draft-mtp,ngram-mod depth 3 / poll 50; idle 22,346–23,300 MiB, ~965 t/s prefill and 257–302 t/s decode at 254k depth; each extra full-context slot costs 3,450 MiB so 4 is the max	the MiMo GGUF path (qwen35 hybrid, asymmetric KV) took the card	re-fetch HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced; era trio in git show 374041c:scripts/frank-small.service / .env / serve-frank-small.sh
2026-09-19 (same-day revert)	gemma-4-12B-it-qat-w4a16-ct 10,264,229,896 B + bf16 assistant 845,719,296 B	vLLM dev image localhost/vllm/vllm-openai:frank-small (8dad46d79c46, still in rootless storage), rootless Quadlet	262,144 window, bf16 KV pool 289,647 tokens, 2 concurrent sessions (the 3rd queues), image + audio; 1,293 t/s prefill @67k falling to 608 @213k, ~13.6 t/s decode @213k depth, MTP acceptance 2.80–3.56, boot 135 s	reverted to llama.cpp the same day	both payloads purged; git show 374041c^:dankclaw/frank-small/Dockerfile and …:dankclaw/frank-small/quadlet/frank-small.container
2026-09-16 → 09-17	ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S + BF16 projector, in-file MTP drafted	mainline llama.cpp	single-slot 3090 shape	superseded by the Gemma4-12B payload	re-download from ISTA-DASLab — no payload and no fetch script survive
2026-09-02 → 09-15 (Dankserver, :7770, the card's previous host)	batsclamp/Huihui-Qwen3.6-35B-A3B-abliterated-FP8 ~35 GB (before it official Qwen3.6-35B-A3B-FP8)	FreeToken v0.1.2 (ft), venv ~/freetoken/.venv	MoE cache 2,048 slots + KV 453,005 tokens + 24 mamba-state slots; ~30 t/s single-stream decode, ~87 t/s at batch 4, ~1,130 t/s prefill at 4k	the 3090 left Dankserver and frank-medium-rocmfpx took the endpoint and port	host units ~/.AGENTTRASH/20260915-frank-zealot-retire/host-units/, repo trio in scripts/.AGENTTRASH/
~2026-08 (Dankserver 3090, pre-FreeToken)	Qwen3.6-35B-A3B Q6_K_P, then Tiel-Coder-35B-A3B UD-Q6_K_XL, with an Ornith Q6_K rung	llama.cpp 555881ebc (b10121) CUDA	--n-cpu-moe 38 deployed (36 recommended), batch 4096 / ubatch 2048, 700,416-token f16 K+V pool (800,768 cells did not fit); ~45.5 t/s decode and 1,188 t/s prefill at ncmoe 35	FreeToken kept experts in host RAM instead of CPU-offloading layers	re-download; nothing from this rung survives on disk

Laws learned here

• Addressable ≠ advertised. The card is 24,576 MiB but leaves 24,123 MiB: quote nvidia-smi's free field, never total − used — the driver's reservation is the difference.
• Exact KV and a 786,432 pool cannot coexist here: 32,768 B/token ⇒ 24,576 MiB before a single weight loads (the retired "26,112 MiB" note doubled q8_0's scale overhead); the 27B analogue died at 25,088 MiB with cudaMalloc failed: out of memory. Halving the pool funds exact KV; the per-slot window stops at the trained 262,144 (no YaRN here), so a third conversation is a KV-dtype decision, never more context. The wall's own figures: cudaMalloc failed on a 25,088.00 MiB KV request when the engine's fit projection asked for 33,198 MiB against 23,558 MiB free. It booted f16 at all because the arm env's invented KV_CACHE_TYPE was swallowed by the serve script's :=f16 default — so a failed window became the measurement.
• Exact KV reads 1.882× the bytes per generated token, not 2× (32,768 vs 17,408 B/token, 2026-10-18): a decode drop of that order is EXPECTED and must be measured, while prefill should hold — and this service can afford the dtype precisely because its load is the compaction hub's ~170k-token prefill, which is weight-bandwidth bound and never re-reads the cache per generated token. The dtype is paid in decode and concurrency.
• Never --fit with parallel > 1 — it maximizes the total pool and shrinks each slot: set CTX_SIZE explicitly and keep it divisible by PARALLEL (the seam enforces it).
• KV rate law. full_layers × 4 KV heads × 256 dim = 8,192 elements per token per cache on this arch (the engine prints n_embd_k_gqa = n_embd_v_gqa = 1024), ×2 B (f16), ×1.0625 (q8_0 — 32 codes plus one 2-byte scale per 32-block), ×0.625 (q4_0): 32,768 / 17,408 / 10,240 B/token on this 8-full-layer 9B, and 34,816 for a 16-full-layer 27B at q8_0. Everything else bills separately: DeltaNet recurrent state per slot (150.75 MiB for 3 cells, and an armed depth-2 drafter triples it to 452.25), compute buffers at graph capture, the mmproj after the fit. On gemma4-class hybrids attention_k_eq_v does not shrink the allocation — the cache reserves the K/V pair twice, 6.125 B/elem at f16 K + q8_0 V.
• --flash-attn on is pinned, not decorative: a quantized V exists only on the FA path and the engine falls back silently without it. llama.cpp has no true fp8 KV dtype — q8_0 is the 8-bit class (1.0625 B/elem, ~8.5 effective bits). GGML_CUDA_FA_ALL_QUANTS is vestigial and build-specific: the PrismML fork carries it ON (since 2026-09-17) while the master build had it OFF and still dispatched Q4/Q5 KV unconditionally (ggml-cuda/fattn.cu has no guard), so quantized KV never required it.
• Prices come from boots, not formulas, and an arm belongs to the payload it was measured on: 3 × 180,224 + q8_0 V fits with Q5_K_M (21,684 used / 2,441 free) and does not fit with Q6_K inside the 2–3 GiB band. The mmproj worst case (~870 MiB) is printed before fitting but never billed by the engine's own fit check. Only tests/frank_zealot_unit.bats's fit gate can see a boot that would die at cudaMalloc, so every retune keeps it green — and the first verbosity-4 ledger plus settled nvidia-smi of a new dtype belongs on this page, because projections drift. The two boots that priced non-KV: 2 × 262,144 without the projector — 20,190 used, KV 12,800 ⇒ non-KV 7,390 MiB; 3 × 196,608 with it — 22,972 used, KV 14,113 ⇒ non-KV 8,859 MiB. The difference prices the vision tower at ~1.4 GiB for a 918 MB file, because the encoder's activation and compute buffers ride with it. The first 3-slot deploy (3 × 196,608) booted healthy and left only 1,153 MiB free — that is how the 2–3 GiB band got violated in practice, and why the seam prices non-KV at the measured 9 GiB and refuses that shape.
• Ubatch is a prefill knob. Decode never moves with it and the logical --batch-size is neutral; prefill climbs monotonically to 2048 on this CUDA card (+4.6 % at 8.9k depth; 710 → 1,188 t/s), a prompt shorter than the ubatch cannot see the knob at all (the control row), and a verdict never crosses backends — the 2× R9700 split peaks at 512. Each rung costs ~0.6–0.75 GiB of compute buffers; 2048 is the ceiling for a 27B-on-3090 shape.
• A quant rung is paid in decode: Q5_K_M → Q6_K is +875 MiB of weights and −10 % rate on this bandwidth-bound card, and the byte-bandwidth law predicts a rung's rate within 1 %.
• Speculation pays only with room on the card. The 2026-10-18 arm census (all q8_0 K+V at the 786,432 pool): 3 × 262,144 spec-OFF ubatch 2048 → 17,876 / 6,249 free, 167.2 t/s; spec-ON ubatch 2048 → 22,706 / 1,419, 109.2 t/s; spec-ON ubatch 1024 → 21,400 / 2,725, 213.5 t/s; 2 × 262,144 spec-ON → 213.2; 1 × 262,144 spec-ON → 214.2. That is +28 % decode whenever the card has room, and the doors back are one window per conversation, or q8_0 KV with ubatch 1024. The draft's own bill at that shape was 3,072.00 MiB of KV plus ~1,758 MiB of buffers.
• Draft KV is built against the target's whole pool (~2.4 GiB measured on the 27B Swift boot, not the ~0.4 GiB a proposal-depth estimate suggests). Spec flags move as a SET — SPEC_TYPE=none strips all of them and half a set has segfaulted at load. Two context-backed drafters cannot share a process, so draft-mtp,ngram-mod is the only legal pair, and on a qwen35 hybrid ngram-mod disables recurrent-state rollback so rejected drafts degrade silently.
• Acceptance alone never justifies speculation: 2026-08-25 measured 26.8–35.2 t/s drafting against 53.7 no-spec.
• A checkpoint declaring *_num_hidden_layers MTP while carrying zero mtp.*/nextn.* tensors boots a 4.24 GiB ghost drafter at accept rate: 0.00 — read the safetensors index, not the config.
• MTP is a property of the file: the plain PQ2_0 sibling could not draft at any flag value; only the -MTP export carries blk.N.nextn.*.
• The format is the runtime lock. PQ2_0 is general.file_type 141 over 202 tensors of custom GGML type 142 plus prism.hadamard.* metadata: stock llama.cpp refuses the type, and a loader ignoring the Hadamard transform would dequant wrong, silently (same lock class as ROCmFPX). Build ≥ b10776 is additionally the floor that pairs MTP with a quantized V cache.
• Check arch support in the source tree: grep -c on the packed binary returns 0 for every arch, including the one it serves.
• Base alignment. Bonsai-2 is the censored base lineage. /home/dank/models/Bonsai2-CRACK/ matches the 27B geometry (a one-line MODEL_FILE swap there, identical tensor census); no dealigned 9B exists, so for this payload the refusal escape hatch is a fetch, not a revert.
• A stale bench drop-in hijacks the production boot and no health check sees it. An Environment=FRANK_ZEALOT_ENV_FILE=/tmp/<arm>.env in frank-zealot.service.d/ booted a 27B PTQ1_0 payload while /health answered 200, /v1/models echoed the alias flag and the e2e gate passed 33/33 — none of them read the payload.
• After any bench window: systemctl show -p DropInPaths → tr '\0' ' ' </proc/$pid/cmdline → ledger, in that order; drop-ins retire to ~/.AGENTTRASH/<date>-<topic>/, and payload decisions land on this page the moment they are made, or an approved arm becomes indistinguishable from a rogue one a day later.
• An arm env must use the serve script's own variable names (CACHE_TYPE_K/CACHE_TYPE_V, BIND_ADDR, NGL): an invented KV_CACHE_TYPE is swallowed by : "${CACHE_TYPE_K:=f16}" and the arm boots f16 — an OOM that reads like a model fault.
• Stop, shebang, GPU node. llama.cpp rides out the default stop timeout (TimeoutStopSec=180 is on the unit for that reason) and a hand-booted arm keeps answering 7771 behind a failed unit — kill it by PID, never pkill -f llama-server.
• serve-frank-zealot.sh is a direct ExecStart, so a missing shebang is a 203/EXEC boot loop at Mem peak: 1.9M with the GPU untouched: head -1 is a deploy check.
• No CUDA GPUs are available ⇒ nvidia-modprobe -u -c=0; KV spilled into unified memory halves decode silently, so re-read nvidia-smi after any shape change.

Contract

• /health → {"status":"ok"}; /props reports n_ctx and presence_penalty; /slots shows occupancy; a third concurrent conversation waits for a slot — the slot count is the concurrency contract.
• /metrics answers 200 with 15 llamacpp:* families only with --metrics; without it llama.cpp answers 501, which Prometheus reads as DOWN while /health stays 200. The scrape target is host.containers.internal:7771/metrics and the oldest retained sample of up{job="frank-zealot"} (2026-10-11) is already 0 — this target was never successfully scraped, across every payload era. METRICS=0 is the documented opt-out and re-breaks it; the counters cost nothing on the request path.
• Reasoning arrives in reasoning_content (--reasoning-format deepseek) and counts against max_tokens: a short budget yields empty content with finish_reason: "length", which reads as a broken model, not a mid-thought one. The slot prefix cache is live (cached_tokens in usage).
• A client that sends no sampler is not greedy — the engine applies the payload's own general.sampling.* (1.0 / 0.95 / 20). presence_penalty is the one recipe value with no GGUF metadata key, so it must reach argv; the serve script uses ${PRESENCE_PENALTY=1.0}, deliberately not :=, because "" is the documented opt-out.
• Vision: only the OpenAI image_url content part reaches the model — the legacy top-level images field bumps usage.prompt_tokens while the model invents an answer. Probe with a ≥ 256 px solid color (a postage stamp answers "video") and budget ≥ 512 tokens; a 512 px image bills ~280 prompt tokens.
• Q8_0 mmproj, not BF16: the publisher warns the BF16 tower "can crash llama-server when several image requests run at once", which is exactly the --parallel 2 shape. MMPROJ_FILE= (explicitly empty) is the text-only opt-out and returns ~1.4 GiB; a path at a missing file is a boot refusal, never silent blindness.
• Tools: native via --jinja; finish_reason: tool_calls with valid JSON arguments.
• Two non-faults: a 400 naming 262,144 is one conversation outgrowing its slot, not pool exhaustion; and the raw /completion path is a dead instrument on this fork build — a greedy ignore_eos run answers 500 "The model produced output that does not match the expected Content-only format", because build 10777 guards the raw path the moment reasoning markup leaks (older builds only logged failed to parse reasoning content and answered anyway). The honest per-request instrument here is /v1/chat/completions, whose non-stream response carries the same timings object (prompt_n, prompt_per_second, predicted_n, predicted_per_second). Keep VERBOSITY=4 — 3 is the engine default and cannot prove the KV or draft sizes, while levels above 4 flood the journal.
• llama.cpp does not validate the requested model id: it answers and echoes the served ALIAS, so a rename breaks id-keyed consumers (the OpenWebUI model row, its access_grant, openai.api_configs) and nothing server-side. Alias history: frank-small → frank-secondary (2026-10-06) → frank-zealot (7c77326, 2026-10-09); the service and unit took the name 2026-10-10 (55f87fc).
• The first request after a boot is a warmup artifact (37 t/s class) and is never a measurement; never time a wall-clock aggregate on a generation that stops on its own (use ignore_eos).

Files

• scripts/frank-zealot.env — the authority for the values above and nothing else: as of 2026-10-18 it carries the knobs plus the one-line reason a reader editing that line cannot recover here. The arm table, the reference boot ledger and the KV derivation all live on this page now; the header used to hold them, and two of its rung figures were wrong (CACHE_TYPE_V=q8_0 and CTX_SIZE=401408 both free 3,840 MiB at the 524,288 pool, not 8,192 and not ~6.7 GiB — the Fallback rungs table above is the corrected statement).
• scripts/serve-frank-zealot.sh — the exec line and its rationale comments; honors FRANK_ZEALOT_ENV_FILE for arm boots; exits 1 naming a missing binary, GGUF or projector rather than half-loading (a restart policy would turn an unfinished download into a crash loop that reads like a runtime fault).
• scripts/frank-zealot.service — root unit, relinked into /etc/systemd/system/.
• scripts/download-frank-zealot-bonsai9.sh (active) · …-bonsai2.sh · …-bonsai2-ptq1.sh · scripts/download-frank-zealot.sh (master-era GGUFs) · old/download-frank-zealot-sglang-int8.sh (SGLang).
• tests/frank_zealot_unit.bats — stub-argv seam plus the VRAM fit gate.
• scripts/deploy-remote.sh — deploy_llm_zealot (payload-path gates, MainPID-hardened health); see deployment.
• shared/prometheus/prometheus.yml, shared/prometheus/rules/llm-fleet.yml, shared/grafana/provisioning/dashboards/llm-fleet.json — the scrape, the alert, the dashboard.
• Live state, not repo files: the OpenWebUI picker row (back up webui.db before re-keying) and ~/.omp/agent/models.yml.