inference: frank-large returns to halogen, frank-small takes the 3090

Dankclaw's endpoints are re-split. frank-large is the halogen-flash-server
quadlet again -- the 2026-09-13 unit file was never touched by the two-day
ik_llama.cpp interlude, so the runtime was still in the repo and its image is
still in local storage -- retuned to the shape asked for: HALOGEN_CTX 200192
with a 400,384-position pool and two slots, exactly two 200k conversations.

frank-small is new: mainline llama.cpp on the RTX 3090 serving
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF at IQ3_S, using the -mtp build (it carries
blk.64.nextn.*, which is what lets --spec-type draft-mtp run on the main
model's own weights instead of a second 12 GiB copy) plus the BF16 vision
projector, as one 131,072-token slot with f16 KV on port 7771.

The host is shared now, and that is what the halogen retune pays for:
HALOGEN_MAX_TOK 24576 (arena 20.6 -> 16.4 GiB, ~4-5% prefill) and the vision
tower dropped (1.97 GiB) hand ~6 GiB back to the host, and the reserve rises to
20 to keep the fit check honest. Vision moved wholesale to frank-small.

Retired to scripts/.AGENTTRASH/: the ik trio (env, service, serve script), the
Qwen-Sharp template only that runtime read, and four benchmark harnesses bound
to retired runtimes. Because retiring a file makes any test that greps it pass
by absence, the seams assert the archive exists AND the live path does not.

scripts/frank-small.env

@@ -0,0 +1,113 @@
+# Tunables for frank-small (ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF, the IQ3_S
+# build with the MTP head, plus the BF16 vision projector) on mainline
+# llama.cpp, Dankclaw's RTX 3090, since 2026-09-16.
+#
+# Runtime: llama.cpp master 37b53fd (2026-09-16), CUDA build for sm_86:
+#   cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON \
+#     -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86
+#   cmake --build build-cuda --target llama-server -j
+# GGML_CUDA_FA_ALL_QUANTS is deliberately OFF in that build: it is only needed
+# for QUANTIZED KV with flash attention, and this service runs f16 KV (below).
+#
+# Model: ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF, the `-mtp` build of IQ3_S.
+# GSQ-RCO assigns a separate quantization type to every tensor from a
+# gradient-based search under a total size budget, rather than applying one
+# type throughout; IQ3_S at 3.50 bpw is the release's task-lossless operating
+# point (AIME25 100.00 and LiveCodeBench v6 85.71 match BF16 exactly,
+# GPQA-Diamond 0.51 behind, at 11.8 GB against 53.8).
+#
+# WHY THAT FILE: the `-mtp` build is ~0.35 GB larger because it carries the
+# model's Multi-Token Prediction head as GGUF block `blk.64.nextn.*`
+# (qwen35.nextn_predict_layers = 1). llama.cpp loads it from the SAME file --
+# `--spec-type draft-mtp` sets load_mtp, and the draft context runs on the main
+# model's weights (common/common.cpp passes shares_model when there is no
+# --model-draft; speculative.cpp gives it LLAMA_CONTEXT_TYPE_MTP), so there is
+# no second 12 GB copy in VRAM. MTP is therefore a property of the FILE, not of
+# the flags: the plain IQ3_S build has no MTP tensors to load.
+#
+# Architecture (GGUF arch qwen35): 64 layers laid out as 16 x (3 x (Gated
+# DeltaNet -> FFN) + 1 x (Gated Attention -> FFN)) -- 48 linear-attention
+# layers with CONSTANT-SIZE per-sequence state and 16 full-attention layers,
+# 4 KV heads x 256 head_dim. Only the attention layers take KV, so a token
+# costs 2 (K,V) x 4 heads x 256 x 2 B = 64 KiB at f16 (34 KiB at q8_0, 18 KiB
+# at q4_0), and the DeltaNet layers add ~0.16 GiB per sequence in f32 at any
+# depth.
+#
+# SIZING (RTX 3090, 24576 MiB) -- modelled, and the loader's own ledger
+# ("KV self size", "compute buffer size") outranks it:
+#   weights                            11.29 GiB (12,120,016,960 B)
+#   mmproj                              0.87 GiB (931,146,528 B, 27 blocks)
+#   MTP draft KV                        0.50 GiB (its context is n_ctx long
+#                                       over one nextn layer: 4 KiB/token)
+#   DeltaNet state                      0.16 GiB (one sequence)
+#   CUDA context + prefill/decode bufs  ~1.5 GiB
+#   KV at CTX_SIZE                      8.00 GiB (131,072 x 64 KiB)
+#                                      --------
+#                                      ~22.3 GiB of 24, ~2 GiB of slack
+# The binding number is the KV: ~9.5 GiB is left after the fixed terms, which
+# at 64 KiB/token is ~155k tokens across all slots. The deployed shape is one
+# slot at 131,072, full f16 precision, no KV quantization risk, no rebuild.
+# Alternatives inside the same budget: 2 x 65536 at f16; 4 x 65536 or
+# 1 x 262144 at q8_0, both of which need a rebuild with
+# -DGGML_CUDA_FA_ALL_QUANTS=ON (without it quantized-KV prefill collapses
+# ~12x on CUDA -- see the CUDA lesson in AGENTS.md).
+# --parallel N DIVIDES --ctx-size between slots; it does not multiply. One slot
+# means a second concurrent request queues behind the first.
+#
+# Sampling: greedy by default, which is the fleet contract (a request that
+# sends no temperature decodes greedy; a client that sends one gets its own).
+# Qwen's own agentic evaluations used temperature 1.0 / top_p 0.95, and the
+# harness applies exactly that per model in models.yml (extraBody), the same
+# way it did for the earlier frank-large runtime.
+#
+# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (HIP reads
+# those bare vars on the ROCm hosts -- see the 2026-08-13 incident in
+# AGENTS.md). This box is CUDA, but the rule keeps env files portable.
+
+# llama.cpp runtime binary: mainline master 37b53fd, CUDA sm_86.
+LLAMA_BIN=/home/dank/GitPackages/llama.cpp-mainline/build-cuda/bin/llama-server
+
+# Model package. The -mtp file is required for speculative decoding.
+MODEL_DIR=/home/dank/models/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp
+MODEL_FILE=Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf
+MMPROJ_PATH=/home/dank/models/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp/mmproj-Qwen3.8-27B-BF16.gguf
+
+# Fleet identity and network. 7771 is this service's own endpoint on Dankclaw;
+# 7770 stays frank-large's. The alias is the id at /v1/models and what clients
+# ask for -- the harness entry in models.yml names it.
+ALIAS=frank-small
+BIND_ADDR=0.0.0.0
+LISTEN_PORT=7771
+
+# One slot at 131,072 tokens with f16 KV (see the sizing note above).
+CTX_SIZE=131072
+PARALLEL=1
+CACHE_TYPE_K=f16
+CACHE_TYPE_V=f16
+
+# Batch geometry: llama.cpp's own defaults (2048 logical / 512 physical). The
+# prefill wall clock at this model size is set by the card, not by ubatch, and
+# the small ubatch keeps the DeltaNet chunk buffers small. Raise UBATCH_SIZE
+# only after reading the compute buffer size the loader reports.
+BATCH_SIZE=2048
+UBATCH_SIZE=512
+
+# CPU threads: this service runs entirely on the GPU (NGL=999), so threads
+# cover tokenization, sampling and the HTTP layer only. Deliberately modest --
+# this host's CPU is shared with halogen's engine, which is CPU-hungry.
+NUM_THREADS=8
+THREAD_BATCH=8
+
+# Every layer on the 3090, with flash attention.
+NGL=999
+FLASH_ATTN=on
+
+# Speculative decoding from the model's own MTP head (see the model note).
+# Draft depth stays llama.cpp's default of 3 tokens.
+SPEC_TYPE=draft-mtp
+
+# Greedy default (fleet contract); reasoning is extracted into
+# reasoning_content with an unlimited budget (the model decides).
+TEMPERATURE=0.0
+REASONING_FORMAT=deepseek
+REASONING_BUDGET=-1