Frank
7d3dd33de0
Frank's order: replace frank-large with kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF's Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf, following that card. The file is ftype 114 — 339 GGML_TYPE_Q6_0_ROCMFPX (102) + 159 GGML_TYPE_Q8_0_ROCMFPX (103) + 353 F32 over 851 tensors, qwen35/64 layers/full_attention_interval 4/4 KV heads/262,144 window — read out of its own header. The pinned ROCmFPX build already knows both types (ggml.h 102/103 plus convert/cpy/getrows/mmq/fattn device paths for 102), so no rebuild: the fork stays at 36e9acd and the payload directory is the only new host state. Card compliance, all already our flags: Q4_0 draft head (not Q8_0), --spec-draft-n-max 4 (the knee on both cards), --spec-draft-ngl 99, --jinja, --fit off, -fa on, -dio, one slot at full window, thinking with the card's own sampling triple (which the file also embeds as its general.sampling defaults). Kept from the deployed shape: the BF16 projector (this repo ships only a Q8_0 one; BF16 is the file verified 4/4 on spatial ground truth here) and the byte-identical MTP head, which the new script hardlinks instead of re-fetching 1.6 GiB. Franksdank-Runbook: serverAppConfigs-docs/LLM/frank-large-rocmfpx-q6agent-uncensored-mtp.md
117 lines
4.6 KiB
Bash
Executable File
117 lines
4.6 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Download kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF for
|
|
# frank-large (Dankclaw iGPU). Public repo — no auth header, and never a token
|
|
# on the command line (it is world-readable in /proc/<pid>/cmdline).
|
|
#
|
|
# The payload is the repo's Q6 AGENT rung:
|
|
# * Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf — ftype 114, 23.15 GiB.
|
|
# Q6_0_ROCMFPX tensor type (102) + the AGENT routing, which keeps the
|
|
# output-head and attention tensors at plain Q8_0 — the variant the card
|
|
# names for MTP serving ("Take Q6 AGENT (114) if you want protected
|
|
# heads"). Like every ROCmFPX file it loads on nothing but the fork.
|
|
#
|
|
# The other two files frank-large needs are NOT re-downloaded:
|
|
# * mtp-Qwen3.8-27B-Q4_0.gguf — the uncensored repo redistributes the
|
|
# ALIGNED repo's draft head byte for byte (identical Hub LFS oid
|
|
# 051a1764…), so it is hardlinked out of the aligned payload dir when that
|
|
# dir holds the verified bytes, and fetched from this repo otherwise.
|
|
# * mmproj-Qwen3.8-27B-BF16.gguf — this repo ships a Q8_0 projector; the
|
|
# fleet keeps the BF16 one, which is the projector that was verified 4/4
|
|
# on spatial ground truth on this host. Hardlinked the same way, fetched
|
|
# from the ALIGNED repo if it is not on disk.
|
|
#
|
|
# Fresh downloads, no resume (`curl -C -` once produced an oversized file on a
|
|
# 31 GiB payload); -f/--fail-with-body so an HTTP 401 is not a success, then
|
|
# byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
|
|
#
|
|
# Usage:
|
|
# scripts/download-frank-large-kingjones-uncensored.sh [DEST_DIR] [--verify-only]
|
|
set -euo pipefail
|
|
|
|
REPO="kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF"
|
|
ALIGNED_REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
|
|
BASE="https://huggingface.co/${REPO}/resolve/main"
|
|
ALIGNED_BASE="https://huggingface.co/${ALIGNED_REPO}/resolve/main"
|
|
SOURCE_DIR="${SOURCE_DIR:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP}"
|
|
|
|
DEST=""
|
|
VERIFY_ONLY=0
|
|
for arg in "$@"; do
|
|
case "$arg" in
|
|
--verify-only) VERIFY_ONLY=1 ;;
|
|
*) DEST="$arg" ;;
|
|
esac
|
|
done
|
|
DEST="${DEST:-/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP}"
|
|
|
|
# name bytes sha256 base_url may_link — may_link=1 means the same bytes may
|
|
# already exist in SOURCE_DIR (hardlink them instead of re-fetching).
|
|
FILES=(
|
|
"Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf 24856607008 e966deb3d1a5a1b9ca7765126292f2d822f3dccdedb3b4872e673dec1cddc199 $BASE 0"
|
|
# The draft head is redistributed byte-identical by both repos; the BF16
|
|
# projector only exists in the aligned repo.
|
|
"mtp-Qwen3.8-27B-Q4_0.gguf 1680271648 051a1764cff8c4f3ee6ae8b00593a0364c7539c67fa50ffc58f3f96509fca38e $BASE 1"
|
|
"mmproj-Qwen3.8-27B-BF16.gguf 931145888 de2a49866988ea272c43dc2a43ee2662c25725a717ca5b9059ea420a97f53fe8 $ALIGNED_BASE 1"
|
|
)
|
|
|
|
matches() { # matches FILE SIZE SHA — byte count AND LFS sha256
|
|
[[ -f "$1" ]] && [[ "$(stat -c%s "$1")" == "$2" ]] \
|
|
&& sha256sum -c --quiet <(printf '%s %s\n' "$3" "$1")
|
|
}
|
|
|
|
mkdir -p "$DEST"
|
|
status=0
|
|
|
|
fetch() { # fetch NAME SIZE SHA BASE_URL
|
|
local name="$1" size="$2" sha="$3" url="$4" f="$DEST/$1"
|
|
echo "FETCHING $name"
|
|
curl -fsSL --fail-with-body -o "$f.part" "$url/$name"
|
|
mv "$f.part" "$f"
|
|
if [[ "$(stat -c%s "$f")" != "$size" ]]; then
|
|
echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
|
|
return 1
|
|
fi
|
|
if ! sha256sum -c --quiet <(printf '%s %s\n' "$sha" "$f"); then
|
|
echo "SHA FAIL $name"
|
|
return 1
|
|
fi
|
|
echo "VERIFIED $name"
|
|
}
|
|
|
|
for entry in "${FILES[@]}"; do
|
|
read -r name size sha url may_link <<<"$entry"
|
|
f="$DEST/$name"
|
|
if matches "$f" "$size" "$sha"; then
|
|
echo "OK $name"
|
|
continue
|
|
fi
|
|
if (( VERIFY_ONLY )); then
|
|
echo "MISSING $name"
|
|
status=1
|
|
continue
|
|
fi
|
|
# Shared bytes: prefer the local copy (same filesystem, so a hardlink costs
|
|
# nothing) over re-downloading gigabytes that hash identically.
|
|
if [[ "$may_link" == 1 && -n "${SOURCE_DIR:-}" ]] && matches "$SOURCE_DIR/$name" "$size" "$sha"; then
|
|
echo "LINKING $name from $SOURCE_DIR"
|
|
ln -f "$SOURCE_DIR/$name" "$f" 2>/dev/null || cp -f "$SOURCE_DIR/$name" "$f"
|
|
if matches "$f" "$size" "$sha"; then
|
|
echo "VERIFIED $name"
|
|
continue
|
|
fi
|
|
echo "SHA FAIL $name (linked copy)"
|
|
status=1
|
|
continue
|
|
fi
|
|
if ! fetch "$name" "$size" "$sha" "$url"; then
|
|
status=1
|
|
fi
|
|
done
|
|
|
|
if (( status == 0 )); then
|
|
echo "DOWNLOAD_DONE $DEST"
|
|
else
|
|
echo "DOWNLOAD_FAILED $DEST" >&2
|
|
fi
|
|
exit "$status"
|