Frank
8029119b19
Frank's call. Payload: ajgazin's abliterated Swift-1.5 UD-Q8_K_XL (29.3 GiB, SWIFT OPEN LICENSE) -> unsloth/Qwen3.8-27B-GGUF UD-Q4_K_XL (16.35 GiB, sha256 3f227079...b01e, vanilla/Apache-2.0). NOTE: the endpoint is no longer uncensored — that was one of the reasons for the Swift payload on 2026-09-28. New fetcher scripts/download-frank-large-q4.sh (no auth header: repo is public and ungated; size + sha256 verified after transfer). The speculative drafter and its fetcher are unchanged. Shape: --sessions 4 --context 163840 with unquantized KV. gufo exposes no KV quantization lever (execution_policy.hpp pins kv_cache_storage to kFp16), so f16 is inherent. KV arithmetic: f16 KV = 64 KiB/token (16 full-attention layers x 4 KV heads x 256 head_dim x 2 x 2B; the other 48 layers are linear-attention with a per-sequence fp32 state). 4 x 163840 = 655,360 tokens = 40.0 GiB KV, each session exactly 10 GiB. Budget ~62 GiB of the 114 GiB GTT. Seam retargeted (FETCH -> the q4 script, TARGET_STUB, context/sessions pins).
77 lines
3.1 KiB
Bash
Executable File
77 lines
3.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Download the Q4_K_XL frank-large payload on Dankclaw:
|
|
# unsloth/Qwen3.8-27B-GGUF -> Qwen3.8-27B-UD-Q4_K_XL.gguf
|
|
#
|
|
# bash scripts/download-frank-large-q4.sh
|
|
# bash scripts/download-frank-large-q4.sh --verify-only
|
|
#
|
|
# Integrity: the file is checked against its declared byte count and sha256
|
|
# (the LFS oid from the Hub tree API, 2026-10-01). NO AUTH HEADER IS USED:
|
|
# the repo is public and ungated (gated: false), and a token on the curl
|
|
# command line is world-readable via /proc/<pid>/cmdline. If this ever needs
|
|
# a token, put it in a mode-600 file and pass it with `curl -K`.
|
|
#
|
|
# Layout after a successful run (MODEL_DIR):
|
|
# Qwen3.8-27B-UD-Q4_K_XL.gguf 17559178144 (16.35 GiB)
|
|
#
|
|
# WHY Q4 AND NOT THE Q8 SWIFT PAYLOAD: this is the vanilla (non-abliterated)
|
|
# Qwen3.8-27B at UD-Q4_K_XL, ~13 GiB smaller than the Swift UD-Q8_K_XL. Every
|
|
# GiB of weights saved is a GiB of f16 KV the endpoint can hold instead: the
|
|
# KV costs 64 KiB/token (16 full-attention layers x 4 KV heads x 256 head_dim
|
|
# x 2 (K,V) x 2 B), so 4 sessions x 163840 ctx = 640,000 tokens = 39.06 GiB.
|
|
#
|
|
# The speculative drafter is UNCHANGED and still fetched by its own script
|
|
# (scripts/download-frank-large.sh, z-lab's Hub repo). That name must not
|
|
# appear here at all — the frank-large seam greps THIS file for it to prove
|
|
# the drafter was not folded into the payload fetch.
|
|
#
|
|
# NOTE ON MTP: this payload has no baked-in MTP head (the Hub repo ships one
|
|
# separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-large does not need it —
|
|
# spec decode here is the block-diffusion drafter, not MTP.
|
|
set -euo pipefail
|
|
|
|
MODEL_DIR="/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL"
|
|
REL="Qwen3.8-27B-UD-Q4_K_XL.gguf"
|
|
SIZE=17559178144
|
|
SHA=3f227079003add2511437e5b1e94812e363385225bf6a9b47b0054a72bc8b01e
|
|
URL="https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/${REL}"
|
|
|
|
mode="download"
|
|
[[ "${1:-}" == "--verify-only" ]] && mode="verify"
|
|
|
|
if [[ "$mode" == "download" ]]; then
|
|
mkdir -p "$MODEL_DIR"
|
|
if [[ -s "$MODEL_DIR/$REL" ]]; then
|
|
echo "already present: $MODEL_DIR/$REL"
|
|
else
|
|
echo "fetching $REL (17559178144 bytes) — this is ~16.4 GiB"
|
|
# -f so an HTTP error is a failure, not a 153-byte "artifact".
|
|
# -C - so an interrupted run resumes instead of restarting.
|
|
curl -fL -C - --retry 5 --retry-delay 3 -o "$MODEL_DIR/$REL" "$URL"
|
|
fi
|
|
fi
|
|
|
|
echo "verifying $MODEL_DIR against declared size + sha256"
|
|
python3 - "$MODEL_DIR" "$REL" "$SIZE" "$SHA" <<'PY'
|
|
import hashlib, os, sys
|
|
model_dir, rel, size, sha = sys.argv[1], sys.argv[2], int(sys.argv[3]), sys.argv[4]
|
|
dest = os.path.join(model_dir, rel)
|
|
if not os.path.isfile(dest):
|
|
print(f" MISSING {rel}")
|
|
sys.exit(1)
|
|
actual = os.path.getsize(dest)
|
|
if actual != size:
|
|
print(f" SIZE MISMATCH {rel}: {actual} != {size}")
|
|
sys.exit(1)
|
|
h = hashlib.sha256()
|
|
with open(dest, "rb") as f:
|
|
for chunk in iter(lambda: f.read(1 << 24), b""):
|
|
h.update(chunk)
|
|
if h.hexdigest() != sha:
|
|
print(f" SHA256 MISMATCH {rel}: {h.hexdigest()}")
|
|
sys.exit(1)
|
|
print(f" OK {rel} ({actual} bytes, sha256 matches the Hub LFS oid)")
|
|
PY
|
|
|
|
echo "Q4_K_XL payload complete: $MODEL_DIR"
|