frank-secondary: swap Gemma4-26B-A4B Q4_K_M -> Qwen3.5-9B-Uncensored Q6_K 2x200,704
Capacity trade: the 9B dense-hybrid (qwen35: 32 layers = 8 full-attention + 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144, MTP-less, natively multimodal) costs ~7 GiB of weights where the 26B MoE cost ~16.6 GiB. The freed room buys back the second conversation slot and exact KV: --ctx-size 401408 --parallel 2 (two 200,704 windows), f16 K + f16 V, MTP off (the file carries no nextn/mtp tensors - flag-only enable is impossible), vision via the publisher's BF16 mmproj, ubatch 1024 (the CUDA decode-optimal value), verbosity 4, publisher sampler recipe (0.6/0.95/20/0) in models.yml. Hosts with the MiMo-V2.6-Distill Q6_K boots at this geometry: 21,276 MiB used / 2,849 free at ubatch 1024, 40,960 tokens of fp16 KV at ubatch 2048. Compaction stays routed here; window math is unchanged (maxTokens 16,384 keeps the worst-case harness request inside the slot). Gemma4 payload (Q4_K_M + mmproj + MTP assistant) stays on disk as the rollback; its runbook is retired in place, keeping the attention_k_eq_v pair-billing law. Co-Authored-By: omp <omp@franksdank.cloud>
This commit is contained in:
1 parent
b51e078e36
commit
1d062f06a3
10 files changed
+360
-213
No files matched your search
@@ -1,27 +1,24 @@
|
||||
#!/usr/bin/env bash
|
||||
# Download the frank-secondary payload for llama.cpp on the RTX 3090:
|
||||
# ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF:RCO-IQ3_XXS
|
||||
# **with the MTP head in the file** (the -mtp variant), plus the
|
||||
# PUBLISHER's OWN vision projector (mmproj-BF16.gguf, same repo). Public
|
||||
# repo — no auth header, and never a token on the command line (it is
|
||||
# world-readable in /proc/<pid>/cmdline).
|
||||
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
|
||||
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
|
||||
# multimodal, so this file is the endpoint's eyes). Public repo — no auth
|
||||
# header, and never a token on the command line (it is world-readable in
|
||||
# /proc/<pid>/cmdline).
|
||||
#
|
||||
# Fresh downloads, no resume (`curl -C -` once produced a 213 MB oversized
|
||||
# file on a 31 GiB payload); -f/--fail-with-body so an HTTP error is not a
|
||||
# success, then byte count and sha256. This repo is on Xet storage, so
|
||||
# the Hub tree/blob APIs expose no LFS oid: the sha256 values below were
|
||||
# computed locally after the first verified fetch (byte counts match the
|
||||
# Hub's declared sizes). Idempotent: complete files are skipped. The
|
||||
# projector is stored under a self-documenting dest name; the URL name
|
||||
# and sha stay pinned in the table below.
|
||||
# success, then byte count and sha256. The Hub exposes no LFS oid for this
|
||||
# repo (Xet storage), so the sha256 values below were computed locally after
|
||||
# the first verified fetch — byte counts match the Hub's declared sizes.
|
||||
# A sha of "-" means "first fetch: size-verified, sha printed for pinning".
|
||||
# Idempotent: complete files are skipped.
|
||||
#
|
||||
# 2026-10-06 (Frank, evening): the payload moved to the UNCELERSED
|
||||
# Swift-1.5-Qwen3.8-27B at RCO-IQ3_XXS (1.56 GiB lighter than the
|
||||
# IQ3_S-mtp it replaced; the lighter quant funds the MTP draft ctx plus
|
||||
# the second 131,072 slot). Rollback payload stays on disk:
|
||||
# /home/dank/models/Swift15-IQ3S/ (the IQ3_S-mtp that was live until this
|
||||
# swap); earlier payloads Ornith-Q8/ and MiMo-Q6K/ as well. Their
|
||||
# re-fetch recipes are in git history (predecessors of this script).
|
||||
# 2026-10-07 (Frank): the payload moved to Qwen3.5-9B-Uncensored Q6_K at
|
||||
# 2 x 200,704 (qwen35 hybrid: 32 layers, 8 full-attention + 24 Gated
|
||||
# DeltaNet, no MTP tensors in this file). Rollback payloads stay on disk:
|
||||
# /home/dank/models/Gemma4-26B-A4B/ (Q4_K_M, the 2026-10-07 era being
|
||||
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
|
||||
@@ -35,15 +32,14 @@ for arg in "$@"; do
|
||||
*) DEST="$arg" ;;
|
||||
esac
|
||||
done
|
||||
DEST="${DEST:-/home/dank/models/Swift15-UncIQ3XXS}"
|
||||
DEST="${DEST:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K}"
|
||||
|
||||
# repo | hub-name | dest-name | bytes | sha256 (computed locally; Xet repo)
|
||||
FILES=(
|
||||
"ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF|RCO/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf|Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf|10442827392|e13d3e7400b6c7f980ebd30c0edd6be43eb1865b50be8562a9db042706c7d803"
|
||||
"ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF|mmproj-BF16.gguf|mmproj-BF16.gguf|931145888|19acd7fcf4ee504328566eddf0f8e603ef853a205b4830209d2a4908213abdea"
|
||||
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|7359259008|c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986"
|
||||
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|921704448|05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934"
|
||||
)
|
||||
|
||||
mkdir -p "$DEST"
|
||||
status=0
|
||||
for entry in "${FILES[@]}"; do
|
||||
IFS='|' read -r repo hubname destname size sha <<<"$entry"
|
||||
@@ -51,7 +47,9 @@ for entry in "${FILES[@]}"; do
|
||||
f="$DEST/$destname"
|
||||
have=0
|
||||
if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
|
||||
if sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||
if [[ "$sha" == "-" ]]; then
|
||||
have=1
|
||||
elif sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||
have=1
|
||||
fi
|
||||
fi
|
||||
@@ -73,6 +71,10 @@ for entry in "${FILES[@]}"; do
|
||||
status=1
|
||||
continue
|
||||
fi
|
||||
if [[ "$sha" == "-" ]]; then
|
||||
echo "VERIFIED $destname (size only) sha256=$(sha256sum "$f" | cut -d' ' -f1)"
|
||||
continue
|
||||
fi
|
||||
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
||||
echo "SHA FAIL $destname"
|
||||
status=1
|
||||
|
||||
Reference in new issue
Block a user