FrankandClaude
7689092db8
Replace the Kairic Edge ROCmFPX build with the unsloth b11030 stock llama.cpp prebuilt (ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF: UD-Q8_K_XL (a 27B dense model, 31.5 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) on the gfx1151 iGPU. Serving shape: one 262,144-token conversation, q8_0 KV (the build lacks fp8), text-only, greedy sampling, Qwen-Sharp v22.4.0 template. - scripts/serve-frank-large.sh: stock llama.cpp + DFlash2 spec decode - scripts/frank-large.env: new model/drafter/runtime, 262144 ctx / 1 slot - scripts/download-frank-large.sh: per-file fetch, verify mode, idempotent - scripts/frank-large.service: updated description - scripts/deploy-remote.sh: updated pkill pattern + asset gates - tests/frank_large_unit.bats: new model/runtime/spec assertions - docs: new runbook (active), Kairic Edge runbook retired (rollback target) Co-Authored-By: Claude <noreply@anthropic.com>
90 lines
3.2 KiB
Bash
Executable File
90 lines
3.2 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Download the frank-large payload on Dankclaw:
|
|
# - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF)
|
|
# - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter)
|
|
#
|
|
# bash scripts/download-frank-large.sh
|
|
# bash scripts/download-frank-large.sh --verify-only
|
|
#
|
|
# Integrity: every file is checked against its declared byte count and sha256
|
|
# (the values below are the LFS oids from the Hub, 2026-09-21). An HF token
|
|
# at ~/.cache/huggingface/token is used when present to clear the anonymous
|
|
# rate limit on a 31 GB transfer.
|
|
#
|
|
# Layout after a successful run (MODEL_DIR):
|
|
# Qwen3.8-27B-UD-Q8_K_XL.gguf 31457991680 main GGUF
|
|
# Qwen3.8-27B-DFlash2-Q4_K_M.gguf 1143006816 DFlash2 drafter
|
|
set -euo pipefail
|
|
|
|
MODEL_DIR="/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL"
|
|
|
|
mode="download"
|
|
[[ "${1:-}" == "--verify-only" ]] && mode="verify"
|
|
|
|
token=""
|
|
auth=()
|
|
if [[ -r "$HOME/.cache/huggingface/token" ]]; then
|
|
token="$(tr -d '[:space:]' < "$HOME/.cache/huggingface/token")"
|
|
auth=(-H "Authorization: Bearer $token")
|
|
fi
|
|
|
|
dl() { # dl <url> <dest>
|
|
local url="$1" dest="$2"
|
|
[[ -s "$dest" ]] && return 0
|
|
mkdir -p "$(dirname "$dest")"
|
|
echo "fetching $dest"
|
|
curl -fL -C - "${auth[@]}" --retry 5 --retry-delay 3 -o "$dest" "$url"
|
|
}
|
|
|
|
# filename size sha256 url — the LFS oids from the Hub (2026-09-21)
|
|
FILES=(
|
|
"Qwen3.8-27B-UD-Q8_K_XL.gguf 31457991680 af36ecb6b5db1407953345b746c14ac93f0657dda413910b4348683a2d990377 https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/Qwen3.8-27B-UD-Q8_K_XL.gguf"
|
|
"Qwen3.8-27B-DFlash2-Q4_K_M.gguf 1143006816 1a25c56858e1ebe93f2718ac1d49d1151f9323325c1bbfd6209370f4db131ebd https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2-GGUF/resolve/main/Qwen3.8-27B-DFlash2-Q4_K_M.gguf"
|
|
)
|
|
|
|
# Verify each file's byte count and sha256 (idempotent: a complete file is
|
|
# re-checked, not re-downloaded).
|
|
echo "verifying $MODEL_DIR against declared sizes + sha256"
|
|
python3 - "$MODEL_DIR" <<'PY'
|
|
import hashlib, os, sys
|
|
model_dir = sys.argv[1]
|
|
# (rel, size, sha256) triples — the LFS oids from the Hub (2026-09-21).
|
|
specs = {
|
|
"Qwen3.8-27B-UD-Q8_K_XL.gguf": (31457991680, "af36ecb6b5db1407953345b746c14ac93f0657dda413910b4348683a2d990377"),
|
|
"Qwen3.8-27B-DFlash2-Q4_K_M.gguf": (1143006816, "1a25c56858e1ebe93f2718ac1d49d1151f9323325c1bbfd6209370f4db131ebd"),
|
|
}
|
|
ok = True
|
|
for rel, (size, sha) in specs.items():
|
|
dest = os.path.join(model_dir, rel)
|
|
if not os.path.isfile(dest):
|
|
print(f" MISSING {rel}")
|
|
ok = False
|
|
continue
|
|
actual_size = os.path.getsize(dest)
|
|
if actual_size != size:
|
|
print(f" SIZE MISMATCH {rel}: {actual_size} != {size}")
|
|
ok = False
|
|
continue
|
|
h = hashlib.sha256()
|
|
with open(dest, "rb") as f:
|
|
for chunk in iter(lambda: f.read(1 << 24), b""):
|
|
h.update(chunk)
|
|
actual_sha = h.hexdigest()
|
|
if actual_sha != sha:
|
|
print(f" SHA256 MISMATCH {rel}: {actual_sha}")
|
|
ok = False
|
|
continue
|
|
print(f" OK {rel} ({actual_size} bytes)")
|
|
if not ok:
|
|
sys.exit(1)
|
|
PY
|
|
|
|
if [[ "$mode" == "download" ]]; then
|
|
for line in "${FILES[@]}"; do
|
|
read -r rel _ _ url <<<"$line"
|
|
dl "$url" "$MODEL_DIR/$rel"
|
|
done
|
|
fi
|
|
|
|
echo "frank-large payload complete: $MODEL_DIR"
|