frank-secondary: swap Gemma4-26B-A4B Q4_K_M -> Qwen3.5-9B-Uncensored Q6_K 2x200,704

Capacity trade: the 9B dense-hybrid (qwen35: 32 layers = 8 full-attention
+ 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144, MTP-less,
natively multimodal) costs ~7 GiB of weights where the 26B MoE cost
~16.6 GiB. The freed room buys back the second conversation slot and
exact KV: --ctx-size 401408 --parallel 2 (two 200,704 windows), f16 K +
f16 V, MTP off (the file carries no nextn/mtp tensors - flag-only enable
is impossible), vision via the publisher's BF16 mmproj, ubatch 1024
(the CUDA decode-optimal value), verbosity 4, publisher sampler recipe
(0.6/0.95/20/0) in models.yml.

Hosts with the MiMo-V2.6-Distill Q6_K boots at this geometry: 21,276 MiB
used / 2,849 free at ubatch 1024, 40,960 tokens of fp16 KV at ubatch
2048. Compaction stays routed here; window math is unchanged (maxTokens
16,384 keeps the worst-case harness request inside the slot).

Gemma4 payload (Q4_K_M + mmproj + MTP assistant) stays on disk as the
rollback; its runbook is retired in place, keeping the attention_k_eq_v
pair-billing law.

Co-Authored-By: omp <omp@franksdank.cloud>

scripts/serve-frank-secondary.sh

@@ -1,27 +1,27 @@
-#!/usr/bin/env bash
-# Serve HauhauCS/Gemma4-26B-A4B-QAT-Uncensored (QAT **Q4_K_M**) as
+# Serve HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q6_K) as
 # frank-secondary on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
 #
-# 2026-10-07 (Frank): Gemma4-26B-A4B swap with MTP OFF and ONE 200,704-token
-# conversation (--parallel 1) — the 24 GiB card cannot fund a second slot at
-# this weight, and the MTP assistant's KV (built against the target's full
-# window, ~1.9 GiB measured) plus the f16-K raise do not coexist on it.
-# Verbosity 4 ships so the VRAM-fit forensics land in the journal.
+# 2026-10-07 (Frank): Qwen3.5-9B swap at TWO 200,704-token conversations
+# (--ctx-size 401408 --parallel 2) with EXACT f16 KV — the 9B rung frees the
+# room the 26B MoE Gemma4 shape lacked, so quantized V retires. MTP stays off:
+# this GGUF carries no MTP tensors at all (verified from its tensor-name
+# list), so there is no in-file head to draft with. Verbosity 4 ships so the
+# VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in the journal.
 #
-# Architecture: gemma4_26b_a4b — 30-layer MoE (active ~4B), hybrid attention:
-# 25 sliding-window layers (fixed 1024-token ring each) + 5 full-attention
-# layers (5, 11, 17, 23, 29), 8 KV heads x 256 dim, n_ctx_train 262,144. KV
-# therefore scales ONLY with the 5 full layers: ~30.7 KiB/token at f16 K +
-# q8_0 V (~6.2 GiB at 200,704). attention_k_eq_v is declared by the file and
-# honoured by the build (bill K-side accounting; the measured boot is the
-# referee: 21,890 used / 2,235 free at q8_0/q8_0, MTP off).
-# Vision comes from the projector file (the GGUF carries no vision
-# tensors; the publisher ships its own BF16 mmproj in the same repo), so
-# --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is the
-# documented text-only opt-out.
+# Architecture: qwen35 — dense 9B, 32 layers with full_attention_interval 4
+# (8 full-attention layers at 3,7,...,31 + 24 Gated DeltaNet linear layers),
+# 16 heads / 4 KV heads × 256 dim, native 262,144, vocab 248,320. The KV pool
+# scales with the 8 full layers: 32 KiB/token at f16 K + f16 V (12.25 GiB at
+# the 401,408 pool); the DeltaNet layers cost a fixed recurrent state per
+# slot. Feasibility is priced from the MiMo-V2.6-Distill-Qwen-9B boots (same
+# geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279
+# at ubatch 2048 — the boot ledger, not arithmetic, is the referee.
+# Vision comes from the publisher's own BF16 projector (the model is natively
+# multimodal), so --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is
+# the documented text-only opt-out.
 #
-# Sampling: llama.cpp takes it per request (the fleet harness sends its
-# recipe from models.yml); greedy is the harness default.
+# Sampling: llama.cpp takes it per request (the fleet harness sends the
+# publisher recipe from models.yml: thinking temp 0.6 / top_p 0.95 / top_k 20).
 #
 # Tunables come from frank-secondary.env unless FRANK_SECONDARY_ENV_FILE overrides.
 set -euo pipefail
@@ -37,32 +37,35 @@ fi
 
 # --- defaults (used when the env file does not set them) -------------------
 : "${LLAMA_BIN:=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
-: "${MODEL_FILE:=/home/dank/models/Gemma4-26B-A4B/Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf}"
+: "${MODEL_FILE:=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf}"
 : "${ALIAS:=frank-secondary}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7771}"
-# ctx-size is the POOL llama.cpp divides across slots; with --parallel 1 the
-# pool IS the conversation window. 200,704 is inside the model's trained
-# 262,144; the harness contextWindow MUST quote this number — a longer prompt
-# is answered with a 400, never truncated.
-: "${CTX_SIZE:=200704}"
-: "${PARALLEL:=1}"
+# ctx-size is the POOL llama.cpp divides across slots: 401,408 / 2 = 200,704
+# per conversation, inside the model's trained 262,144. The harness
+# contextWindow MUST quote the PER-SLOT number — a longer prompt is answered
+# with a 400, never truncated.
+: "${CTX_SIZE:=401408}"
+: "${PARALLEL:=2}"
 # Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
-# 2026-10-07 (Frank): 1024 — cut from 2048 to fund the f16-K raise. The
-# logical batch is VRAM- and throughput-neutral at a fixed ubatch.
+# 2048 is the measured CUDA optimum (MiMo era, same arch/shape: +4.6 % prefill
+# at 8.9k depth, decode flat, 616 MiB). The logical batch is VRAM- and
+# throughput-neutral at a fixed ubatch.
 : "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=1024}"
-# KV at FP8 class = q8_0 on llama.cpp (1.0625 B/elem => 8.5 effective bits;
-# there is no true fp8 KV dtype). Explicit either way so the contract is
-# visible in the argv.
-: "${CACHE_TYPE_K:=q8_0}"
-: "${CACHE_TYPE_V:=q8_0}"
-# Speculative decoding is OFF by default since the Gemma4 swap (Frank,
-# 2026-10-07): Gemma4's MTP is a SEPARATE assistant file (arch
-# gemma4-assistant), not an in-file head, and its KV is built against the
-# target's full window — ~1.9 GiB measured, which the f16-K raise cannot
-# afford. SPEC_TYPE=none strips BOTH speculative flags (flags move as a
-# set); naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE.
+: "${UBATCH_SIZE:=2048}"
+# Exact (unquantized) KV: f16 K + f16 V, restoring the MiMo-era contract that
+# the 26B Gemma4 shape could not afford. 32 KiB/token at this geometry
+# (8 full-attention layers × 4 KV heads × 256 dim); 12.25 GiB at the 401,408
+# pool. Explicit either way so the contract is visible in the argv, and
+# --flash-attn stays required for any quantized-V fallback (q8_0 = llama.cpp's
+# FP8 class; no true fp8 KV dtype exists in this build).
+: "${CACHE_TYPE_K:=f16}"
+: "${CACHE_TYPE_V:=f16}"
+# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors, so
+# there is no in-file head (the law: MTP is a property of the file, not of the
+# flags). SPEC_TYPE=none strips BOTH speculative flags (flags move as a set);
+# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
+# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
 : "${SPEC_TYPE:=none}"
 : "${SPEC_DRAFT_N_MAX:=3}"
 : "${SPEC_DRAFT_FILE:=}"
@@ -73,7 +76,7 @@ fi
 : "${VERBOSITY:=3}"
 # Vision projector. No colon on purpose: an explicitly empty value means
 # "text-only, on purpose" and must NOT be replaced by the default path.
-: "${MMPROJ_FILE=/home/dank/models/Gemma4-26B-A4B/mmproj-Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-BF16.gguf}"
+: "${MMPROJ_FILE=/home/dank/models/Qwen35-9B-HauhauCS-Q6K/mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf}"
 
 # Fail loudly rather than half-loading: a missing file is a download that
 # has not finished, and Restart=always would turn it into a crash loop that