frank-large: swap to stock llama.cpp serving Qwen3.8-27B UD-Q8_K_XL with DFlash2
Replace the Kairic Edge ROCmFPX build with the unsloth b11030 stock llama.cpp prebuilt (ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF: UD-Q8_K_XL (a 27B dense model, 31.5 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) on the gfx1151 iGPU. Serving shape: one 262,144-token conversation, q8_0 KV (the build lacks fp8), text-only, greedy sampling, Qwen-Sharp v22.4.0 template. - scripts/serve-frank-large.sh: stock llama.cpp + DFlash2 spec decode - scripts/frank-large.env: new model/drafter/runtime, 262144 ctx / 1 slot - scripts/download-frank-large.sh: per-file fetch, verify mode, idempotent - scripts/frank-large.service: updated description - scripts/deploy-remote.sh: updated pkill pattern + asset gates - tests/frank_large_unit.bats: new model/runtime/spec assertions - docs: new runbook (active), Kairic Edge runbook retired (rollback target) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
1 parent
43b8258061
commit
7689092db8
10 files changed
+414
-306
No files matched your search
Executable
+89
@@ -0,0 +1,89 @@
|
||||
#!/usr/bin/env bash
|
||||
# Download the frank-large payload on Dankclaw:
|
||||
# - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF)
|
||||
# - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter)
|
||||
#
|
||||
# bash scripts/download-frank-large.sh
|
||||
# bash scripts/download-frank-large.sh --verify-only
|
||||
#
|
||||
# Integrity: every file is checked against its declared byte count and sha256
|
||||
# (the values below are the LFS oids from the Hub, 2026-09-21). An HF token
|
||||
# at ~/.cache/huggingface/token is used when present to clear the anonymous
|
||||
# rate limit on a 31 GB transfer.
|
||||
#
|
||||
# Layout after a successful run (MODEL_DIR):
|
||||
# Qwen3.8-27B-UD-Q8_K_XL.gguf 31457991680 main GGUF
|
||||
# Qwen3.8-27B-DFlash2-Q4_K_M.gguf 1143006816 DFlash2 drafter
|
||||
set -euo pipefail
|
||||
|
||||
MODEL_DIR="/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL"
|
||||
|
||||
mode="download"
|
||||
[[ "${1:-}" == "--verify-only" ]] && mode="verify"
|
||||
|
||||
token=""
|
||||
auth=()
|
||||
if [[ -r "$HOME/.cache/huggingface/token" ]]; then
|
||||
token="$(tr -d '[:space:]' < "$HOME/.cache/huggingface/token")"
|
||||
auth=(-H "Authorization: Bearer $token")
|
||||
fi
|
||||
|
||||
dl() { # dl <url> <dest>
|
||||
local url="$1" dest="$2"
|
||||
[[ -s "$dest" ]] && return 0
|
||||
mkdir -p "$(dirname "$dest")"
|
||||
echo "fetching $dest"
|
||||
curl -fL -C - "${auth[@]}" --retry 5 --retry-delay 3 -o "$dest" "$url"
|
||||
}
|
||||
|
||||
# filename size sha256 url — the LFS oids from the Hub (2026-09-21)
|
||||
FILES=(
|
||||
"Qwen3.8-27B-UD-Q8_K_XL.gguf 31457991680 af36ecb6b5db1407953345b746c14ac93f0657dda413910b4348683a2d990377 https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/Qwen3.8-27B-UD-Q8_K_XL.gguf"
|
||||
"Qwen3.8-27B-DFlash2-Q4_K_M.gguf 1143006816 1a25c56858e1ebe93f2718ac1d49d1151f9323325c1bbfd6209370f4db131ebd https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2-GGUF/resolve/main/Qwen3.8-27B-DFlash2-Q4_K_M.gguf"
|
||||
)
|
||||
|
||||
# Verify each file's byte count and sha256 (idempotent: a complete file is
|
||||
# re-checked, not re-downloaded).
|
||||
echo "verifying $MODEL_DIR against declared sizes + sha256"
|
||||
python3 - "$MODEL_DIR" <<'PY'
|
||||
import hashlib, os, sys
|
||||
model_dir = sys.argv[1]
|
||||
# (rel, size, sha256) triples — the LFS oids from the Hub (2026-09-21).
|
||||
specs = {
|
||||
"Qwen3.8-27B-UD-Q8_K_XL.gguf": (31457991680, "af36ecb6b5db1407953345b746c14ac93f0657dda413910b4348683a2d990377"),
|
||||
"Qwen3.8-27B-DFlash2-Q4_K_M.gguf": (1143006816, "1a25c56858e1ebe93f2718ac1d49d1151f9323325c1bbfd6209370f4db131ebd"),
|
||||
}
|
||||
ok = True
|
||||
for rel, (size, sha) in specs.items():
|
||||
dest = os.path.join(model_dir, rel)
|
||||
if not os.path.isfile(dest):
|
||||
print(f" MISSING {rel}")
|
||||
ok = False
|
||||
continue
|
||||
actual_size = os.path.getsize(dest)
|
||||
if actual_size != size:
|
||||
print(f" SIZE MISMATCH {rel}: {actual_size} != {size}")
|
||||
ok = False
|
||||
continue
|
||||
h = hashlib.sha256()
|
||||
with open(dest, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(1 << 24), b""):
|
||||
h.update(chunk)
|
||||
actual_sha = h.hexdigest()
|
||||
if actual_sha != sha:
|
||||
print(f" SHA256 MISMATCH {rel}: {actual_sha}")
|
||||
ok = False
|
||||
continue
|
||||
print(f" OK {rel} ({actual_size} bytes)")
|
||||
if not ok:
|
||||
sys.exit(1)
|
||||
PY
|
||||
|
||||
if [[ "$mode" == "download" ]]; then
|
||||
for line in "${FILES[@]}"; do
|
||||
read -r rel _ _ url <<<"$line"
|
||||
dl "$url" "$MODEL_DIR/$rel"
|
||||
done
|
||||
fi
|
||||
|
||||
echo "frank-large payload complete: $MODEL_DIR"
|
||||
Reference in new issue
Block a user