frank-secondary: swap Gemma4-26B-A4B Q4_K_M -> Qwen3.5-9B-Uncensored Q6_K 2x200,704

Capacity trade: the 9B dense-hybrid (qwen35: 32 layers = 8 full-attention
+ 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144, MTP-less,
natively multimodal) costs ~7 GiB of weights where the 26B MoE cost
~16.6 GiB. The freed room buys back the second conversation slot and
exact KV: --ctx-size 401408 --parallel 2 (two 200,704 windows), f16 K +
f16 V, MTP off (the file carries no nextn/mtp tensors - flag-only enable
is impossible), vision via the publisher's BF16 mmproj, ubatch 1024
(the CUDA decode-optimal value), verbosity 4, publisher sampler recipe
(0.6/0.95/20/0) in models.yml.

Hosts with the MiMo-V2.6-Distill Q6_K boots at this geometry: 21,276 MiB
used / 2,849 free at ubatch 1024, 40,960 tokens of fp16 KV at ubatch
2048. Compaction stays routed here; window math is unchanged (maxTokens
16,384 keeps the worst-case harness request inside the slot).

Gemma4 payload (Q4_K_M + mmproj + MTP assistant) stays on disk as the
rollback; its runbook is retired in place, keeping the attention_k_eq_v
pair-billing law.

Co-Authored-By: omp <omp@franksdank.cloud>
This commit is contained in:
Frankandomp committed 2026-10-07 14:47:32 -07:00
1 parent b51e078e36
commit 1d062f06a3
10 files changed
+360 -213

No files matched your search

@@ -1,10 +1,11 @@
# frank-secondary — llama.cpp CUDA, Gemma4-26B-A4B Q4_K_M, 1×200,704 (Dankclaw)
**ACTIVE since 2026-10-07.** RTX 3090, port 7771, root systemd unit
`frank-secondary.service`, served by `scripts/serve-frank-secondary.sh` +
`scripts/frank-secondary.env`. Supersedes
[[frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688]] (Swift-1.5-Qwen3.8-27B
RCO-IQ3_XXS-mtp, 2×114,688, in-file MTP).
**RETIRED 2026-10-07** — same day it went live. Superseded by
[[frank-secondary-llama-qwen35-9b-q6k-2x200704]] (Qwen3.5-9B Q6_K at
2 × 200,704 with exact KV); the payload stays at
`/home/dank/models/Gemma4-26B-A4B/` as the rollback. This page keeps the
Gemma4 KV-billing law (the `attention_k_eq_v` pair-billing measurement) and
the drafter-cost boots, both of which outlive the model.
## Payload