frank-secondary: swap Gemma4-26B-A4B Q4_K_M -> Qwen3.5-9B-Uncensored Q6_K 2x200,704
Capacity trade: the 9B dense-hybrid (qwen35: 32 layers = 8 full-attention + 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144, MTP-less, natively multimodal) costs ~7 GiB of weights where the 26B MoE cost ~16.6 GiB. The freed room buys back the second conversation slot and exact KV: --ctx-size 401408 --parallel 2 (two 200,704 windows), f16 K + f16 V, MTP off (the file carries no nextn/mtp tensors - flag-only enable is impossible), vision via the publisher's BF16 mmproj, ubatch 1024 (the CUDA decode-optimal value), verbosity 4, publisher sampler recipe (0.6/0.95/20/0) in models.yml. Hosts with the MiMo-V2.6-Distill Q6_K boots at this geometry: 21,276 MiB used / 2,849 free at ubatch 1024, 40,960 tokens of fp16 KV at ubatch 2048. Compaction stays routed here; window math is unchanged (maxTokens 16,384 keeps the worst-case harness request inside the slot). Gemma4 payload (Q4_K_M + mmproj + MTP assistant) stays on disk as the rollback; its runbook is retired in place, keeping the attention_k_eq_v pair-billing law. Co-Authored-By: omp <omp@franksdank.cloud>
This commit is contained in:
1 parent
b51e078e36
commit
1d062f06a3
10 files changed
+360
-213
No files matched your search
@@ -1,10 +1,11 @@
|
||||
# frank-secondary — llama.cpp CUDA, Gemma4-26B-A4B Q4_K_M, 1×200,704 (Dankclaw)
|
||||
|
||||
**ACTIVE since 2026-10-07.** RTX 3090, port 7771, root systemd unit
|
||||
`frank-secondary.service`, served by `scripts/serve-frank-secondary.sh` +
|
||||
`scripts/frank-secondary.env`. Supersedes
|
||||
[[frank-secondary-llama-swift15-unc-iq3xxs-mtp-114688]] (Swift-1.5-Qwen3.8-27B
|
||||
RCO-IQ3_XXS-mtp, 2×114,688, in-file MTP).
|
||||
**RETIRED 2026-10-07** — same day it went live. Superseded by
|
||||
[[frank-secondary-llama-qwen35-9b-q6k-2x200704]] (Qwen3.5-9B Q6_K at
|
||||
2 × 200,704 with exact KV); the payload stays at
|
||||
`/home/dank/models/Gemma4-26B-A4B/` as the rollback. This page keeps the
|
||||
Gemma4 KV-billing law (the `attention_k_eq_v` pair-billing measurement) and
|
||||
the drafter-cost boots, both of which outlive the model.
|
||||
|
||||
## Payload
|
||||
|
||||
|
||||
Reference in new issue
Block a user