Frank
374041cd7a
The vLLM detour (a2b39b1, same day) is retired: the Q4_K_M llama.cpp setup
is the requested shape and the vLLM KV-quant options on sm_86 all measured
dead (fp8 hardware-impossible, int8 page-size unification, turboquant
mm-prefix). Root unit restored, vLLM quadlet/Dockerfile/fetch script moved
to .AGENTTRASH, runbook listed as same-day-reverted. Rollback target:
frank-small-vllm-gemma4.
109 lines
3.9 KiB
Bash
Executable File
109 lines
3.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-large
|
|
# model) on Dankclaw.
|
|
#
|
|
# bash scripts/download-frank-large-orca.sh
|
|
# bash scripts/download-frank-large-orca.sh --verify-only
|
|
#
|
|
# The package is 136 GB of Hub-sized transport parts (HF caps new uploads at
|
|
# 50 GB, so the 79 GB main GGUF, the 52 GB PLE payload and the 4 GB MTP head
|
|
# ship as .partNofM fragments). assemble.py (from the repo) concatenates the
|
|
# parts, verifies every part and final SHA-256, and consumes the parts.
|
|
#
|
|
# Integrity: every part is checked against the per-part sha256 in parts.json
|
|
# (fetched from the repo, not trusted from disk), then the assembled files
|
|
# against SHA256SUMS. The repo is public (gated: false); an HF token at
|
|
# ~/.cache/huggingface/token is used when present to clear the anonymous
|
|
# rate limit on a 136 GB transfer.
|
|
#
|
|
# Layout after a successful run (MODEL_DIR):
|
|
# Qwen3.8-Flash-CIRU-STRIX-Orca.gguf 79,397,818,912 main model
|
|
# mtp/Qwen3.8-Flash-CIRU-STRIX-Orca-MTP-Q8_0.gguf 4,135,893,440 draft
|
|
# ple/ple.payload.bin 52,429,053,952 external PLE
|
|
# ple/ple.manifest.json, ple/ple.scale.bf16 PLE metadata
|
|
# vision/mmproj-Qwen3.8-Flash-Orca-F16.mmproj 904,004,320 projector
|
|
# assemble.py, parts.json, SHA256SUMS (kept for --verify-only)
|
|
set -euo pipefail
|
|
|
|
REPO="jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca"
|
|
MODEL_DIR="/home/dank/models/Qwen3.8-Flash-CIRU-STRIX-Orca"
|
|
BASE="https://huggingface.co/$REPO/resolve/main"
|
|
|
|
mode="download"
|
|
[[ "${1:-}" == "--verify-only" ]] && mode="verify"
|
|
|
|
token=""
|
|
auth=()
|
|
if [[ -r "$HOME/.cache/huggingface/token" ]]; then
|
|
token="$(cat "$HOME/.cache/huggingface/token")"
|
|
auth=(-H "Authorization: Bearer $token")
|
|
else
|
|
echo "no HF token at ~/.cache/huggingface/token; downloading anonymously"
|
|
fi
|
|
|
|
dl() { # dl <relpath>
|
|
local dest="$MODEL_DIR/$1"
|
|
mkdir -p "$(dirname "$dest")"
|
|
echo " fetching $1"
|
|
curl -fL -C - -o "$dest" --retry 10 --retry-delay 5 --retry-max-time 21600 \
|
|
"${auth[@]}" "$BASE/$1"
|
|
}
|
|
|
|
if [[ "$mode" == "download" ]]; then
|
|
# Manifest + small files first: the part hashes below come from the
|
|
# freshly fetched parts.json, so a stale/corrupt copy cannot pass.
|
|
dl "assemble.py"
|
|
dl "parts.json"
|
|
dl "SHA256SUMS"
|
|
dl "ple/ple.manifest.json"
|
|
dl "ple/ple.scale.bf16"
|
|
dl "vision/SHA256SUMS"
|
|
dl "vision/mmproj-Qwen3.8-Flash-Orca-F16.mmproj"
|
|
|
|
echo "fetching the five transport parts (136 GB)"
|
|
while IFS= read -r part; do
|
|
dl "$part"
|
|
done < <(python3 -c "
|
|
import json
|
|
for item in json.load(open('$MODEL_DIR/parts.json'))['files']:
|
|
for p in item['parts']:
|
|
print(p['path'])
|
|
")
|
|
fi
|
|
|
|
echo "verifying part sha256 (against the repo parts.json)"
|
|
python3 - "$MODEL_DIR" <<'PY'
|
|
import hashlib, json, sys
|
|
from pathlib import Path
|
|
root = Path(sys.argv[1])
|
|
manifest = json.loads((root / 'parts.json').read_text())
|
|
ok = True
|
|
def digest(p):
|
|
h = hashlib.sha256()
|
|
with p.open('rb') as f:
|
|
for data in iter(lambda: f.read(8 * 1024 * 1024), b''):
|
|
h.update(data)
|
|
return h.hexdigest()
|
|
for item in manifest['files']:
|
|
for p in item['parts']:
|
|
path = root / p['path']
|
|
if not path.exists():
|
|
print(f" MISSING {p['path']}"); ok = False; continue
|
|
if path.stat().st_size != p['bytes']:
|
|
print(f" BAD-SIZE {p['path']}"); ok = False; continue
|
|
if digest(path) != p['sha256']:
|
|
print(f" BAD-SHA {p['path']}"); ok = False; continue
|
|
print(f" OK {p['path']}")
|
|
sys.exit(0 if ok else 1)
|
|
PY
|
|
|
|
if [[ "$mode" == "download" ]]; then
|
|
echo "assembling (consumes verified parts)"
|
|
(cd "$MODEL_DIR" && python3 assemble.py)
|
|
fi
|
|
|
|
echo "verifying assembled files against SHA256SUMS"
|
|
(cd "$MODEL_DIR" && sha256sum -c SHA256SUMS)
|
|
(cd "$MODEL_DIR/vision" && sha256sum -c SHA256SUMS)
|
|
echo "frank-large payload complete: $MODEL_DIR"
|