# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) on the RTX 3090,
# :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file, plus
# the refusal-direction adapter applied at load.
#
# ONE OWNER: shape, payload identity, eras, laws and every measured number
# live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the
# knobs and only the reason a reader editing THIS line cannot recover
# elsewhere. Where the two disagree, this file wins on values.
LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server
# The fork is a LOCK, not a preference: custom tensor type 142 + prism.hadamard.*
# metadata. Stock llama.cpp refuses the file; a loader ignoring the Hadamard
# metadata dequantizes wrong, silently.
MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf
# Q8_0 IS MANDATORY: the publisher states the BF16 tower can crash llama-server
# under concurrent image requests, which is exactly --parallel 3.
MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
# Refusal-direction adapter (a rank-1 LoRA, 3.2 MB) applied at load; it edits
# the compute graph, it is not a second model. Scale 1 is the card's "held-out
# refusals 0 %" point. LORA_FILE="" is the documented opt-out — the plain pack
# again, no rebuild — and LORA_SCALE=0 is the same result through the flag.
# PROVENANCE ASYMMETRY: the adapter's own prism.abliterate.base_gguf names
# Ternary-Bonsai-2-9B-PTQ1_0.gguf as the pack its direction was fitted against,
# which is NOT this packing and is not on this host. The tensor names match, so
# it applies; the strength was calibrated on a sibling quantization, so the
# effect is proven by the endpoint's behaviour, never by the metadata.
LORA_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-Uncensored-LoRA.gguf
LORA_SCALE=1
# Served id — the harness/family name, not a fact about the payload.
# Id-keyed consumers: OpenWebUI (model row, access_grant, api_configs), models.yml.
ALIAS=frank-zealot
BIND_ADDR=0.0.0.0
LISTEN_PORT=7771
# CTX_SIZE is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
# conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
# models.yml contextWindow quotes the PER-SLOT number — a longer prompt is a
# 400, never a truncation. The pool is bought, not granted: K+V at 25,088
# B/token is 14,406.00 MiB here and the MTP draft context bills its own f16 KV
# over the WHOLE pool (4,096 B/position ⇒ 2,352.00 MiB), so the old 786,432
# pool cannot carry a drafter at any KV dtype — 18,816 + 3,072 of cache alone
# against the 24,123 MiB the driver leaves addressable.
CTX_SIZE=602112
PARALLEL=3
# The logical batch is VRAM- and throughput-neutral at a fixed ubatch (the law
# on [[frank-zealot]]); 1024×6 keeps it a clean multiple of the ubatch.
BATCH_SIZE=6144
# Ubatch is a prefill knob (decode never moves with it) and it sets the
# compute-buffer size, which decides whether a drafter fits. 1024 is the rung
# every measured MTP arm here ran on (213.5 t/s against 109.2 at 2048). Its
# VRAM credit is SHAPE-specific: the 786,432-pool arm census showed ~1,306 MiB
# back, this boot measured 2,089.88 MiB of target compute against 2,496.13 at
# ubatch 2048 — 406 MiB back, with the split moving (CUDA0 +257, CUDA_Host
# −663). The drafter is funded by CACHE_TYPE_V below, not by this knob.
UBATCH_SIZE=1024
# K exact, V in the 8-bit class: 16,384 + 8,704 = 25,088 B/token (8,192
# elements per token per cache × 2 B for f16, × 1.0625 for q8_0 — there is no
# true fp8 KV dtype in this build). That 7,680 B/token saved against exact V is
# what funds BOTH the third conversation and the drafter. Quantized V is
# illegal without --flash-attn, which the serve script pins. Rate law and the
# arithmetic: [[frank-zealot]] → Laws.
CACHE_TYPE_K=f16
CACHE_TYPE_V=q8_0
# Speculation ON (Frank, 2026-10-19): the head is in the file
# (qwen35.nextn_predict_layers = 1, blk.32.nextn.*) and rides the MAIN model's
# weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of
# draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for
# by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V
# beside MTP needs fork build >= b10776; LLAMA_BIN is 10777.
# SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and
# every --spec-draft-*); half a set has segfaulted at load.
SPEC_TYPE=draft-mtp
SPEC_DRAFT_N_MAX=2
SPEC_DRAFT_N_MIN=0
SPEC_DRAFT_P_MIN=0
# The sampler recipe lives in the checkpoint (general.sampling.*) and
# llama.cpp reads it itself. Presence penalty is the one recipe value with no
# GGUF metadata key, so it must reach argv. "" is the documented opt-out.
PRESENCE_PENALTY=1.0
NGL=99
# 4 = debug: KV layout, buffer sizes and graph capture land in the journal, and
# that is what VRAM forensics read. Above 4 floods the journal.
VERBOSITY=4
# Without --metrics llama.cpp answers 501 on /metrics, which Prometheus reads
# as DOWN while /health stays 200. 0 opts out and re-breaks the scrape.
METRICS=1