frank-secondary: swap Swift-1.5 RCO-IQ3_XXS -> Gemma4-26B-A4B QAT Q4_K_M

Payload: HauhauCS/Gemma4-26B-A4B-QAT-Uncensored Q4_K_M (16,796,015,520 B)
+ the publisher's BF16 mmproj, /home/dank/models/Gemma4-26B-A4B/.

Shape: ONE 200,704-token slot (--parallel 1; 2x200,704 does not fit the
24 GiB card), f16 K + q8_0 V under flash attention, batch 4096 / ubatch
1024, --log-verbosity 4, MTP OFF (the gemma4-assistant file is on disk,
not loaded: its draft KV is built against the target's full window,
measured +1.9 GiB -> 23,806/319 free, no margin beside the f16-K raise).

Measured boots (nvidia-smi): q8_0 K + q8_0 V / ubatch 2048 / MTP off =
21,890 used / 2,235 free; with the MTP assistant = 23,806 / 319.

Engine facts recorded: gemma4 hybrid attention (25 sliding-window + 5
full-attention layers) so KV scales with 5 layers only, and
attention_k_eq_v is honoured by build 5fc4f3c (pool bills K-only; the
measured q8_0 pool = 2.05 GiB = the K-only figure).

Serve script gained VERBOSITY (+ --log-verbosity in the argv), the
SPEC_DRAFT_FILE drafter path, and defaults/lore retargeted to Gemma4.
deploy-remote.sh payload gates follow the env. The frank-secondary bats
seam is rewritten for the new contract (argv-based) and fixes a
pre-existing vacuity: serve_argv regenerates the test env, so
mutations made before it were silently undone - tests that mutate now
exec through run_stage. frank-secondary is the fleet compaction hub
(200,704 is the only window that fits a worst-case compaction input).

AGENTS.md

@@ -27,7 +27,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 |--------|---------|---------|-------|------|
 | Dankserver | `frank-primary` | **radiance** (StillDeadcode v1.0.7 line; image `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned INDEX digest, 287 MiB, `libr4d` gfx1201 kernels, plain `podman run`, root systemd unit) | `Qwen/Qwen3.8-Flash-Next-FP8` official (172.8 GB, 131 shards, sha256-verified) converted **no-recipe** to `qwen4exp-fp8-native.rad` (177,109 MiB measured = 173.0 GiB) — **native modelopt block-FP8, no requantisation** — on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, `--expert-vs-cache-ratio 0.82` = the 99 %-hit point), **PLE 47.7 GiB pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV 7.32 KiB/token**, **MTP spec decode depth 3**, *…
 | Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the head per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 200,192 ctx** (KV pool 400,384 positions ≈ 10.7 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold) | 7770 |
-| Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF **RCO-IQ3_XXS -mtp** (27B dense qwen3_5 hybrid, 10,442,827,392 B; a per-tensor allocation mix of stock types; the MTP head in the file **drafted** — **spec ON** (draft-mtp depth 3) since 2026-10-06 evening, ngram skipped on the recurrent-rollback law) + the publisher's own **BF16 mmproj** for **vision**, on the RTX 3090, **2 × 114,688** (Frank's 112k cap each since the 2026-10-06 late reshape, cut from 131,072 to fund the **ubatch ladder 512 → 1024 → 2048** — **1,005 MiB free at boot, 763 settled after a vision probe** (encoder buffers resident)) with **q8_0 K + q8_0 V (FP8 class, 34 KiB/token)** under flash attention, batch 4096 / **ubatch 2048** (the ceiling for this shape — 4096 would not fit); **prefill 1,209 t/s @ 3,760 tokens; ~52–58 t/s recipe decode** (MTP acceptance 34–54 %, temp-1.0 noise; decode is ubatch-invariant). Renamed from `frank-small` 2026-10-06 (87721ab). Supersedes the same-morning IQ3_S-mtp payloads: the two-slot no-spec shape and the one-slot MTP shape (both 2026-10-06; payload on disk).|7771|
+| Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Gemma4-26B-A4B-QAT-Uncensored **Q4_K_M** (25.2 B-param MoE, ~4 B active, 16,780,192,888 B reported by the engine; 30 layers = 25 sliding-window + 5 full-attention) + the publisher's own **BF16 mmproj** for **vision**, on the RTX 3090, **ONE 200,704-token slot** (`--parallel 1`; two 200,704 slots do not fit 24 GiB), **f16 K + q8_0 V** under flash attention (the file declares `attention_k_eq_v` and this build honours it — the pool bills K-only), batch 4096 / **ubatch 1024**, `--log-verbosity 4`, **MTP OFF** (the assistant is a SEPARATE file and its draft KV is built against the target's full window — 23,806 / 319 MiB free with it loaded, no margin beside the f16-K raise). Measured q8_0-K boot: **21,890 used / 2,235 free**. Cutover 2026-10-07 from Swift-1.5 RCO-IQ3_XXS 2×114,688. All three endpoints route harness **compaction** here. | 7771 |
 
 Retired endpoints (`frank-npu`, and every prior runtime) are in
 the [runbook index](#runbook-index). The full swap history lives in
@@ -308,9 +308,10 @@ image]`) — `frank-large` returned to vision with the flash-server v2 swap
 27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
 are the engine core; `--mm-max-patches` available),
 and `frank-secondary` through
-the publisher's own `mmproj-BF16.gguf` passed as `--mmproj` — the Swift
+the publisher's own BF16 mmproj passed as `--mmproj` — the Gemma4
 GGUF itself carries no vision tensors. `frank-secondary`'s `contextWindow` is
-**114688** (cut from 131072 on 2026-10-06 late to fund the ubatch ladder to 2048), the
+**200704** (the one-slot window since the 2026-10-07 Gemma4 swap; `--parallel 1`,
+so the pool IS the conversation), the
 per-slot window, not the model's native 262144: the harness must compact
 before llama.cpp answers a longer prompt with a 400. Note also
 that
@@ -417,89 +418,63 @@ that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
 `/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`,
 `GGML_CUDA_FA_ALL_QUANTS=ON`) as a **root systemd unit**
 (`scripts/frank-secondary.service`, `relink_unit`ed into
-`/etc/systemd/system/`), serving `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`
-at **RCO-IQ3_XXS -mtp** (10,442,827,392 B, sha256 `e13d3e74…`) **plus
-the publisher's own `mmproj-BF16.gguf`** (931,145,888 B, sha256
-`19acd7fc…`), both `/home/dank/models/Swift15-UncIQ3XXS/`, on port 7771.
-Cutover 2026-10-05 (evening) on Frank's explicit contract: **128k
-context cap, KV at FP8, vision on, as many 128k contexts as fit**;
-reshaped three times on 2026-10-06 — morning to two 131,072 slots
-no-spec, evening to this UNCERSED RCO-IQ3_XXS payload with MTP back
-on, late the RENAME to **frank-secondary** (87721ab) at two 114,688
-slots with the ubatch ladder climbing 512 → 1024 → 2048 (f2575a1,
-d8da7a9). The
-same-day payloads it supersedes: the LIVE MiMo-V2.6 Q6_K shape and the
-Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but never
-deployed (blocked at the docs gate; the docs debt is what this section
-settles).
+`/etc/systemd/system/`), serving
+`HauhauCS/Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-MTP` at
+**Q4_K_M** (16,796,015,520 B on disk; the engine reports
+16,780,192,888 B, 25,233,142,046 params, vocab 262,144) **plus the
+publisher's own BF16 mmproj** (1,194,827,776 B), both
+`/home/dank/models/Gemma4-26B-A4B/`, on port 7771. Cutover 2026-10-07
+from the Swift-1.5 RCO-IQ3_XXS 2×114,688 shape. The MTP assistant
+(`mtp-gemma-4-26B-A4B-it.gguf`, 251,937,728 B) is on disk, **not loaded**.
 
-**Architecture: qwen3_5 hybrid, bigger** — block_count 65 (64 real layers +
-the blk.64 MTP head), `full_attention_interval 4` ⇒ **16 full-attention +
-48 Gated DeltaNet**, native 262,144 window, dense. KV exists only in the 16
-attention layers (4 KV heads × 256 dim): 64 KiB/token at f16, **34 KiB/token
-at q8_0**. The DeltaNet state is a fixed per-slot buffer (~150 MiB/slot at
-48 linear layers × 48 value heads). Header facts were read from the Hub
-file by HTTP range + struct parse before anything was downloaded.
+**Architecture: gemma4 MoE hybrid** — 30 layers, 128 experts (8 active),
+n_embd 2816, native 262,144 window. Attention is hybrid: **25
+sliding-window layers (window 1024, a fixed ring each) + 5 full-attention
+layers (5, 11, 17, 23, 29)**, 8 KV heads × 256 dim — so the KV pool scales
+with the 5 full layers ONLY: ~10.6 KiB/token at q8_0, ~20.5 KiB/token at
+f16. `attention_k_eq_v: True` — `gemma4.cpp` maps the absent `wv` to
+`Vcur = Kcur`, and this build's KV cache honours it (no V slots allocated:
+the q8_0 pool measured ~2.05 GiB at 200,704, the K-only figure). **Bill
+K-only.**
 
-**The shape (2026-10-06 midnight; Frank's tight-fit order, MTP back on
-in the evening, per-slot cut + ubatch climbed to 2048):** `--ctx-size
-229376 --parallel 2` = **TWO 114,688-token conversations** (Frank's
-112k cap each; `n_ctx_slot = CTX_SIZE / PARALLEL`; cut from 131,072 to
-fund the ubatch rungs), `--flash-attn on` pinned, **q8_0 K + q8_0 V**
-(llama.cpp's FP8 class — no true fp8 KV dtype exists; 8.5 effective
-bits; supersedes the same-day f16/f16 rule written for the Ornith 9B
-fit), all layers on the GPU, `--mmproj` for vision, batch 4096 /
-ubatch 2048. **Speculative decoding is ON** (draft-mtp, depth 3): the
--mtp file's head is drafted against the main model's own weights;
-ngram-mod skipped on purpose — on a qwen35 hybrid ngram disables
-recurrent-state rollback, so a rejected draft corrupts the DeltaNet
-state.
+**The shape (2026-10-07):** `--ctx-size 200704 --parallel 1` = ONE
+conversation (Frank's 200k cap; the harness `contextWindow` quotes
+200704), `--flash-attn on`, **f16 K + q8_0 V** (exact K, FP8-class V),
+batch 4096 / **ubatch 1024** (cut from 2048 to fund the f16-K raise),
+`--log-verbosity 4` (debug — the KV/buffer forensics land in the journal),
+`--mmproj` for vision, **speculative decoding OFF** (`SPEC_TYPE=none`).
 
-**Measured at the midnight boot (2026-10-06, 2 × 114,688 / ubatch
-2048): 23,120 used / 1,005 MiB free, settling 763 MiB after a vision
-probe** (`nvidia-smi` free; the encoder buffers stay resident on this
-boot) — the per-slot cut freed ~1.05 GiB of KV and both ubatch rungs
-spent it back (~0.3 + ~0.75 GiB measured; the 1024 rung had landed
-22,374/1,751, the 2048 rung cost 746 MiB, ~130 over the 616 the rung
-measured on the MiMo shape). The MTP draft ctx costs **~2.4 GiB
-measured** (23,636 − 22,728 + 1,561 at the 262,144 draft window) — ~6×
-the ~0.4 GiB the one-slot era assumed, because the drafter's KV is
-built against the target's FULL context, not the 3-token proposal
-depth; the lighter RCO-IQ3_XXS quant (−1.56 GiB vs IQ3_S-mtp) funds it.
-**Prefill 1,209 t/s @ 3,760 tokens; decode 52.0–58.2 tok/s (recipe
-temp 1.0) / 66.9 (temp 0)**, acceptance 34.2–54.4 % — decode is
-ubatch-invariant, the spread is temp-1.0 acceptance noise.
-`reasoning_effort` probe unchanged: `xhigh` 200, `high` → 500 Jinja
-raise (the models.yml fold stays mandatory).
+**Why MTP is off**: Gemma4's MTP is a SEPARATE assistant file (arch
+`gemma4-assistant`), not an in-file head, and the draft context is built
+against the target's FULL window — measured cost **~1.9 GiB** (23,806 used
+/ 319 free with the assistant loaded, vs 21,890 / 2,235 without). The
+assistant also warns `n_ctx_seq (200704) > n_ctx_train (131072)` — the
+draft runs 53 % past its trained window. The q8_0-K boot leaves 2,235 MiB,
+which the f16-K raise (+2.05 GiB) spends; MTP and exact-K cannot coexist on
+this card.
 
-**Vision with the publisher's own projector**: ajgazin's repo SHIPS
-`mmproj-BF16.gguf` in the same repo (the IQ3_S era's unsloth mmproj was
-built from the SHARED BASE `Qwen/Qwen3.8-27B`, legal because Swift-1.5's
-vision stack was hash-proven byte-identical to the base). The probe
-traps carry over: ≥256 px probe images or the model answers "video";
-only the OpenAI `image_url` content part reaches the model. **New on
-this payload: the uncensored model thinks more than the censored
-Swift 1.5** — a 64-token budget is eaten entirely by
-`reasoning_content` (empty `content`); 128 tokens clears the vision
-probe.
+**Measured boots** (nvidia-smi, the referee): q8_0 K + q8_0 V, ubatch 2048,
+MTP off: **21,890 used / 2,235 free**; same shape with the MTP assistant:
+**23,806 / 319**. The f16-K + ubatch-1024 boot: see the runbook's ledger
+(projected ~23 GiB). Decode is expected ubatch-invariant (Swift-era 3090
+law; unmeasured on Gemma4).
 
-The RCO-IQ3_XXS tier is a **per-tensor allocation mix** of stock quant
-types (IQ3_S 97 / BF16 96 / IQ3_XXS 83 / IQ2_S 72 / IQ4_XS 33 / IQ2_XS 32
-/ IQ2_XXS 30 / Q2_K 28 / Q4_K 19 / Q6_K 8 / IQ1_S 4 / IQ1_M 4) — stock
-llama.cpp loads it; the tensor-type histogram is the law (see Key
-Lessons). Publisher KLD vs Swift BF16: mean 0.002631 / 90th 0.042136 /
-max 34.49 — RCO is the publisher's "more experimental" family, and the
-payload is the UNCECERSED Swift 1.5 (23/100 refusals vs Swift 1.5's
-98/100, KL 0.0884).
+**Vision**: the publisher's own BF16 mmproj via `--mmproj`. Probe traps
+carry over from the Swift era: ≥256 px probe images or the model answers
+"video", and only the OpenAI `image_url` content part reaches the model.
+
+**Compaction hub**: `models.yml` routes `compactionModel` for
+`frank-primary`, `frank-large` and `frank-secondary` here — 200,704 is the
+only window in the fleet that fits a worst-case compaction input (~133k
+tokens). Consequence of `--parallel 1`: compaction requests serialize
+behind live generation on this slot.
 
 Runbook:
-`serverAppConfigs-docs/LLM/frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md`.
-Rollback: `/home/dank/models/Swift15-IQ3S/` (the IQ3_S-mtp payload live
-until the 2026-10-06 evening swap — revert da36fa3 for the two-slot
-no-spec shape) or `/home/dank/models/MiMo-Q6K/` (revert to the
-2×200,704 f16 shape — [[frank-secondary-llama-mimo-q6k-2x200704]]) or
-`/home/dank/models/Ornith-Q8/` (the committed-never-deployed Q8_0;
-revert to 380aafa's tree).
+`serverAppConfigs-docs/LLM/frank-secondary-llama-gemma4-q4km-200704.md`.
+Rollback: revert the swap commit — the Swift payload
+(`/home/dank/models/Swift15-UncIQ3XXS/`), the MiMo Q6_K payload
+(`/home/dank/models/MiMo-Q6K/`) and the Ornith Q8_0 payload
+(`/home/dank/models/Ornith-Q8/`) all remain on disk.
 
 ### frank-primary — Radiance (StillDeadcode), Qwen3.8-Flash-Next FP8 native, TP=2 (Dankserver, ACTIVE)
 
@@ -1022,7 +997,7 @@ Active:
 | Service | Runbook |
 |---------|---------|
 | frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×200,192, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` |
-| frank-secondary (llama.cpp CUDA, Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
+| frank-secondary (llama.cpp CUDA, Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off, Dankclaw) | `serverAppConfigs-docs/LLM/frank-secondary-llama-gemma4-q4km-200704.md` |
 | frank-primary (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -1037,6 +1012,7 @@ Retired (rollback targets and history):
 | FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-primary`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-primary-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-primary-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-primary-freetoken-pr587.md` |
 | llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-secondary-llama-mimo-q6k-2x200704.md` |
 | llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-secondary`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
+| llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-secondary`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688.md` |
 | ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
 | gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
 | vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |