Frankandomp
1d062f06a3
Capacity trade: the 9B dense-hybrid (qwen35: 32 layers = 8 full-attention + 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144, MTP-less, natively multimodal) costs ~7 GiB of weights where the 26B MoE cost ~16.6 GiB. The freed room buys back the second conversation slot and exact KV: --ctx-size 401408 --parallel 2 (two 200,704 windows), f16 K + f16 V, MTP off (the file carries no nextn/mtp tensors - flag-only enable is impossible), vision via the publisher's BF16 mmproj, ubatch 1024 (the CUDA decode-optimal value), verbosity 4, publisher sampler recipe (0.6/0.95/20/0) in models.yml. Hosts with the MiMo-V2.6-Distill Q6_K boots at this geometry: 21,276 MiB used / 2,849 free at ubatch 1024, 40,960 tokens of fp16 KV at ubatch 2048. Compaction stays routed here; window math is unchanged (maxTokens 16,384 keeps the worst-case harness request inside the slot). Gemma4 payload (Q4_K_M + mmproj + MTP assistant) stays on disk as the rollback; its runbook is retired in place, keeping the attention_k_eq_v pair-billing law. Co-Authored-By: omp <omp@franksdank.cloud>
91 lines
3.3 KiB
Bash
Executable File
91 lines
3.3 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Download the frank-secondary payload for llama.cpp on the RTX 3090:
|
|
# HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at **Q6_K** plus the
|
|
# PUBLISHER's OWN vision projector (the BF16 mmproj — the model is natively
|
|
# multimodal, so this file is the endpoint's eyes). Public repo — no auth
|
|
# header, and never a token on the command line (it is world-readable in
|
|
# /proc/<pid>/cmdline).
|
|
#
|
|
# Fresh downloads, no resume (`curl -C -` once produced a 213 MB oversized
|
|
# file on a 31 GiB payload); -f/--fail-with-body so an HTTP error is not a
|
|
# success, then byte count and sha256. The Hub exposes no LFS oid for this
|
|
# repo (Xet storage), so the sha256 values below were computed locally after
|
|
# the first verified fetch — byte counts match the Hub's declared sizes.
|
|
# A sha of "-" means "first fetch: size-verified, sha printed for pinning".
|
|
# Idempotent: complete files are skipped.
|
|
#
|
|
# 2026-10-07 (Frank): the payload moved to Qwen3.5-9B-Uncensored Q6_K at
|
|
# 2 x 200,704 (qwen35 hybrid: 32 layers, 8 full-attention + 24 Gated
|
|
# DeltaNet, no MTP tensors in this file). Rollback payloads stay on disk:
|
|
# /home/dank/models/Gemma4-26B-A4B/ (Q4_K_M, the 2026-10-07 era being
|
|
# replaced), Swift15-UncIQ3XXS/, MiMo-Q6K/, MiMo-Q5KM/, Ornith-Q8/.
|
|
#
|
|
# Usage:
|
|
# scripts/download-frank-secondary.sh [DEST_DIR] [--verify-only]
|
|
set -euo pipefail
|
|
|
|
DEST=""
|
|
VERIFY_ONLY=0
|
|
for arg in "$@"; do
|
|
case "$arg" in
|
|
--verify-only) VERIFY_ONLY=1 ;;
|
|
*) DEST="$arg" ;;
|
|
esac
|
|
done
|
|
DEST="${DEST:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K}"
|
|
|
|
# repo | hub-name | dest-name | bytes | sha256 (computed locally; Xet repo)
|
|
FILES=(
|
|
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf|7359259008|c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986"
|
|
"HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf|921704448|05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934"
|
|
)
|
|
|
|
status=0
|
|
for entry in "${FILES[@]}"; do
|
|
IFS='|' read -r repo hubname destname size sha <<<"$entry"
|
|
url="https://huggingface.co/${repo}/resolve/main/${hubname}"
|
|
f="$DEST/$destname"
|
|
have=0
|
|
if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
|
|
if [[ "$sha" == "-" ]]; then
|
|
have=1
|
|
elif sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
|
have=1
|
|
fi
|
|
fi
|
|
if (( have )); then
|
|
echo "OK $destname"
|
|
continue
|
|
fi
|
|
if (( VERIFY_ONLY )); then
|
|
echo "MISSING $destname"
|
|
status=1
|
|
continue
|
|
fi
|
|
echo "FETCHING $destname (from $repo)"
|
|
tmp="$f.part"
|
|
curl -fsSL --fail-with-body -o "$tmp" "$url"
|
|
mv "$tmp" "$f"
|
|
if [[ "$(stat -c%s "$f")" != "$size" ]]; then
|
|
echo "SIZE FAIL $destname ($(stat -c%s "$f") != $size)"
|
|
status=1
|
|
continue
|
|
fi
|
|
if [[ "$sha" == "-" ]]; then
|
|
echo "VERIFIED $destname (size only) sha256=$(sha256sum "$f" | cut -d' ' -f1)"
|
|
continue
|
|
fi
|
|
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
|
echo "SHA FAIL $destname"
|
|
status=1
|
|
continue
|
|
fi
|
|
echo "VERIFIED $destname"
|
|
done
|
|
if (( status == 0 )); then
|
|
echo "DOWNLOAD_DONE $DEST"
|
|
else
|
|
echo "DOWNLOAD_FAILED $DEST" >&2
|
|
fi
|
|
exit "$status"
|