frank-secondary: swap Swift-1.5 RCO-IQ3_XXS -> Gemma4-26B-A4B QAT Q4_K_M

Payload: HauhauCS/Gemma4-26B-A4B-QAT-Uncensored Q4_K_M (16,796,015,520 B)
+ the publisher's BF16 mmproj, /home/dank/models/Gemma4-26B-A4B/.

Shape: ONE 200,704-token slot (--parallel 1; 2x200,704 does not fit the
24 GiB card), f16 K + q8_0 V under flash attention, batch 4096 / ubatch
1024, --log-verbosity 4, MTP OFF (the gemma4-assistant file is on disk,
not loaded: its draft KV is built against the target's full window,
measured +1.9 GiB -> 23,806/319 free, no margin beside the f16-K raise).

Measured boots (nvidia-smi): q8_0 K + q8_0 V / ubatch 2048 / MTP off =
21,890 used / 2,235 free; with the MTP assistant = 23,806 / 319.

Engine facts recorded: gemma4 hybrid attention (25 sliding-window + 5
full-attention layers) so KV scales with 5 layers only, and
attention_k_eq_v is honoured by build 5fc4f3c (pool bills K-only; the
measured q8_0 pool = 2.05 GiB = the K-only figure).

Serve script gained VERBOSITY (+ --log-verbosity in the argv), the
SPEC_DRAFT_FILE drafter path, and defaults/lore retargeted to Gemma4.
deploy-remote.sh payload gates follow the env. The frank-secondary bats
seam is rewritten for the new contract (argv-based) and fixes a
pre-existing vacuity: serve_argv regenerates the test env, so
mutations made before it were silently undone - tests that mutate now
exec through run_stage. frank-secondary is the fleet compaction hub
(200,704 is the only window that fits a worst-case compaction input).

scripts/serve-frank-secondary.sh

@@ -1,23 +1,22 @@
 #!/usr/bin/env bash
-# Serve ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
-# (RCO-IQ3_XXS, **MTP head in the file**) as frank-secondary on Dankclaw's
-# RTX 3090 via llama.cpp (master build, CUDA sm_86).
+# Serve HauhauCS/Gemma4-26B-A4B-QAT-Uncensored (QAT **Q4_K_M**) as
+# frank-secondary on Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
 #
-# 2026-10-06 (Frank): TWO 131,072 conversations (262144 / 2) at the 128k
-# cap, ubatch 512, and MTP BACK ON (draft-mtp, depth 3 — the payload is
-# 1.56 GiB lighter than the IQ3_S-mtp it replaced, which is what funds
-# the draft ctx plus the second slot; ngram stays skipped — on a qwen35
-# hybrid it disables recurrent-state rollback). The q8_0 KV contract
-# (FP8 class) is unchanged; the per-tensor flags stay explicit either
-# way.
+# 2026-10-07 (Frank): Gemma4-26B-A4B swap with MTP OFF and ONE 200,704-token
+# conversation (--parallel 1) — the 24 GiB card cannot fund a second slot at
+# this weight, and the MTP assistant's KV (built against the target's full
+# window, ~1.9 GiB measured) plus the f16-K raise do not coexist on it.
+# Verbosity 4 ships so the VRAM-fit forensics land in the journal.
 #
-# Architecture: qwen35 hybrid — 64 real layers, 16 full-attention + 48
-# Gated DeltaNet (full_attention_interval 4), native 262144 window, plus
-# the blk.64 MTP head (nextn_predict_layers=1). KV lives only in the 16
-# attention layers (34 KiB/token at q8_0); the DeltaNet state is per-slot
-# fixed size.
+# Architecture: gemma4_26b_a4b — 30-layer MoE (active ~4B), hybrid attention:
+# 25 sliding-window layers (fixed 1024-token ring each) + 5 full-attention
+# layers (5, 11, 17, 23, 29), 8 KV heads x 256 dim, n_ctx_train 262,144. KV
+# therefore scales ONLY with the 5 full layers: ~30.7 KiB/token at f16 K +
+# q8_0 V (~6.2 GiB at 200,704). attention_k_eq_v is declared by the file and
+# honoured by the build (bill K-side accounting; the measured boot is the
+# referee: 21,890 used / 2,235 free at q8_0/q8_0, MTP off).
 # Vision comes from the projector file (the GGUF carries no vision
-# tensors; the publisher SHIPS mmproj-BF16.gguf in the same repo), so
+# tensors; the publisher ships its own BF16 mmproj in the same repo), so
 # --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is the
 # documented text-only opt-out.
 #
@@ -38,40 +37,43 @@ fi
 
 # --- defaults (used when the env file does not set them) -------------------
 : "${LLAMA_BIN:=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
-: "${MODEL_FILE:=/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf}"
+: "${MODEL_FILE:=/home/dank/models/Gemma4-26B-A4B/Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf}"
 : "${ALIAS:=frank-secondary}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7771}"
-# ctx-size is the POOL llama.cpp divides across slots: 262144 / 2 gives
-# TWO 131,072 conversations — Frank's 128k cap each. 2026-10-06 (Frank):
-# drop MTP, drop ubatch to 512, take two slots; the fit is tight by his
-# explicit order (frank-secondary.env carries the arithmetic and the floor
-# violation).
-: "${CTX_SIZE:=262144}"
-: "${PARALLEL:=2}"
+# ctx-size is the POOL llama.cpp divides across slots; with --parallel 1 the
+# pool IS the conversation window. 200,704 is inside the model's trained
+# 262,144; the harness contextWindow MUST quote this number — a longer prompt
+# is answered with a 400, never truncated.
+: "${CTX_SIZE:=200704}"
+: "${PARALLEL:=1}"
 # Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
-# Frank's 2026-10-06 tight-fit order ships 4096/512 (claws back ~0.3 GiB
-# toward the second slot); ubatch 1024 is the first A/B back if a future
-# boot lands with headroom (a miss fails loud at graph capture). The
+# 2026-10-07 (Frank): 1024 — cut from 2048 to fund the f16-K raise. The
 # logical batch is VRAM- and throughput-neutral at a fixed ubatch.
 : "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=512}"
+: "${UBATCH_SIZE:=1024}"
 # KV at FP8 class = q8_0 on llama.cpp (1.0625 B/elem => 8.5 effective bits;
 # there is no true fp8 KV dtype). Explicit either way so the contract is
 # visible in the argv.
 : "${CACHE_TYPE_K:=q8_0}"
 : "${CACHE_TYPE_V:=q8_0}"
-# Speculative decoding is ON by default (Frank, 2026-10-06 evening):
-# draft-mtp against the head that is IN the file. SPEC_TYPE none strips
-# BOTH speculative flags (flags move as a set) — a no-spec baseline must
-# not carry half a spec pair.
-: "${SPEC_TYPE:=draft-mtp}"
+# Speculative decoding is OFF by default since the Gemma4 swap (Frank,
+# 2026-10-07): Gemma4's MTP is a SEPARATE assistant file (arch
+# gemma4-assistant), not an in-file head, and its KV is built against the
+# target's full window — ~1.9 GiB measured, which the f16-K raise cannot
+# afford. SPEC_TYPE=none strips BOTH speculative flags (flags move as a
+# set); naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE.
+: "${SPEC_TYPE:=none}"
 : "${SPEC_DRAFT_N_MAX:=3}"
 : "${SPEC_DRAFT_FILE:=}"
 : "${NGL:=99}"
+# Log verbosity threshold (llama.cpp -lv). 3 is the engine default (info);
+# 4 adds debug detail (KV layout, buffer sizes, graph capture) which is what
+# the VRAM-fit forensics need. Levels above 4 flood the journal.
+: "${VERBOSITY:=3}"
 # Vision projector. No colon on purpose: an explicitly empty value means
 # "text-only, on purpose" and must NOT be replaced by the default path.
-: "${MMPROJ_FILE=/home/dank/models/Swift15-UncIQ3XXS/mmproj-BF16.gguf}"
+: "${MMPROJ_FILE=/home/dank/models/Gemma4-26B-A4B/mmproj-Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-BF16.gguf}"
 
 # Fail loudly rather than half-loading: a missing file is a download that
 # has not finished, and Restart=always would turn it into a crash loop that
@@ -118,6 +120,7 @@ ARGS=(
     --flash-attn on
     --cache-type-k "$CACHE_TYPE_K"
     --cache-type-v "$CACHE_TYPE_V"
+    --log-verbosity "$VERBOSITY"
     --jinja
     --reasoning auto
     --reasoning-format deepseek