frank-secondary: swap Gemma4-26B-A4B Q4_K_M -> Qwen3.5-9B-Uncensored Q6_K 2x200,704

Capacity trade: the 9B dense-hybrid (qwen35: 32 layers = 8 full-attention
+ 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144, MTP-less,
natively multimodal) costs ~7 GiB of weights where the 26B MoE cost
~16.6 GiB. The freed room buys back the second conversation slot and
exact KV: --ctx-size 401408 --parallel 2 (two 200,704 windows), f16 K +
f16 V, MTP off (the file carries no nextn/mtp tensors - flag-only enable
is impossible), vision via the publisher's BF16 mmproj, ubatch 1024
(the CUDA decode-optimal value), verbosity 4, publisher sampler recipe
(0.6/0.95/20/0) in models.yml.

Hosts with the MiMo-V2.6-Distill Q6_K boots at this geometry: 21,276 MiB
used / 2,849 free at ubatch 1024, 40,960 tokens of fp16 KV at ubatch
2048. Compaction stays routed here; window math is unchanged (maxTokens
16,384 keeps the worst-case harness request inside the slot).

Gemma4 payload (Q4_K_M + mmproj + MTP assistant) stays on disk as the
rollback; its runbook is retired in place, keeping the attention_k_eq_v
pair-billing law.

Co-Authored-By: omp <omp@franksdank.cloud>
This commit is contained in:
Frankandomp committed 2026-10-07 14:47:32 -07:00
1 parent b51e078e36
commit 1d062f06a3
10 files changed
+360 -213

No files matched your search

+25 -23
View File
@@ -1,27 +1,24 @@
#!/usr/bin/env bash
# Download the frank-secondary payload for llama.cpp on the RTX 3090:
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF:RCO-IQ3_XXS
# **with the MTP head in the file** (the -mtp variant), plus the
# PUBLISHER's OWN vision projector (mmproj-BF16.gguf, same repo). Public
# repo — no auth header, and never a token on the command line (it is
# world-readable in /proc/<pid>/cmdline).
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
# multimodal, so this file is the endpoint's eyes). Public repo — no auth
# header, and never a token on the command line (it is world-readable in
# /proc/<pid>/cmdline).
#
# Fresh downloads, no resume (`curl -C -` once produced a 213 MB oversized
# file on a 31 GiB payload); -f/--fail-with-body so an HTTP error is not a
# success, then byte count and sha256. This repo is on Xet storage, so
# the Hub tree/blob APIs expose no LFS oid: the sha256 values below were
# computed locally after the first verified fetch (byte counts match the
# Hub's declared sizes). Idempotent: complete files are skipped. The
# projector is stored under a self-documenting dest name; the URL name
# and sha stay pinned in the table below.
# success, then byte count and sha256. The Hub exposes no LFS oid for this
# repo (Xet storage), so the sha256 values below were computed locally after
# the first verified fetch — byte counts match the Hub's declared sizes.
# A sha of "-" means "first fetch: size-verified, sha printed for pinning".
# Idempotent: complete files are skipped.
#
# 2026-10-06 (Frank, evening): the payload moved to the UNCELERSED
# Swift-1.5-Qwen3.8-27B at RCO-IQ3_XXS (1.56 GiB lighter than the
# IQ3_S-mtp it replaced; the lighter quant funds the MTP draft ctx plus
# the second 131,072 slot). Rollback payload stays on disk:
# /home/dank/models/Swift15-IQ3S/ (the IQ3_S-mtp that was live until this
# swap); earlier payloads Ornith-Q8/ and MiMo-Q6K/ as well. Their
# re-fetch recipes are in git history (predecessors of this script).
# 2026-10-07 (Frank): the payload moved to Qwen3.5-9B-Uncensored Q6_K at
# 2 x 200,704 (qwen35 hybrid: 32 layers, 8 full-attention + 24 Gated
# DeltaNet, no MTP tensors in this file). Rollback payloads stay on disk:
# /home/dank/models/Gemma4-26B-A4B/ (Q4_K_M, the 2026-10-07 era being
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
#
# Usage:
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
@@ -35,15 +32,14 @@ for arg in "$@"; do
*) DEST="$arg" ;;
esac
done
DEST="${DEST:-/home/dank/models/Swift15-UncIQ3XXS}"
DEST="${DEST:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K}"
# repo | hub-name | dest-name | bytes | sha256 (computed locally; Xet repo)
FILES=(
"ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF|RCO/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf|Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf|10442827392|e13d3e7400b6c7f980ebd30c0edd6be43eb1865b50be8562a9db042706c7d803"
"ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF|mmproj-BF16.gguf|mmproj-BF16.gguf|931145888|19acd7fcf4ee504328566eddf0f8e603ef853a205b4830209d2a4908213abdea"
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|7359259008|c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986"
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|921704448|05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934"
)
mkdir -p "$DEST"
status=0
for entry in "${FILES[@]}"; do
IFS='|' read -r repo hubname destname size sha <<<"$entry"
@@ -51,7 +47,9 @@ for entry in "${FILES[@]}"; do
f="$DEST/$destname"
have=0
if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
if sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
if [[ "$sha" == "-" ]]; then
have=1
elif sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
have=1
fi
fi
@@ -73,6 +71,10 @@ for entry in "${FILES[@]}"; do
status=1
continue
fi
if [[ "$sha" == "-" ]]; then
echo "VERIFIED $destname (size only) sha256=$(sha256sum "$f" | cut -d' ' -f1)"
continue
fi
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
echo "SHA FAIL $destname"
status=1