frank-medium: serve the new R9700s, retire frank-small
The RTX 3090 left Dankserver; two AMD Radeon AI PRO R9700 cards (Navi 48, gfx1201, 32,624 MiB each) replaced it. FreeToken/CUDA cannot run there at all, so the `frank-small` service is retired and `frank-medium` takes the endpoint. New service: charlie12345/ROCmFPX llama.cpp (built for gfx1201 against the fork's self-contained ROCm 7.14 toolchain, so the host's stale /opt/rocm-6.3.3 never participates) serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF -- a Q8_0_ROCMFPX_AGENT (ftype 115) checkpoint that stock llama.cpp cannot load. Placeholder config: --split-mode tensor across both cards, 131072 context, 1 slot, f16 KV, MTP draft head on ROCm0. deploy-remote.sh grows retire_root_llm_small(): the CUDA-era unit is stopped, disabled and moved to ~/.AGENTTRASH/20260915-frank-small-retire/host-units before frank-medium starts, because `systemctl disable` has been observed to delete the unit file itself on this host (2026-08-26 incident). The retired frank-small repo files (env, unit, serve script) move to scripts/.AGENTTRASH/, and the planner, dispatcher, service allow-list and the deployment test seams now name frank-medium.
This commit is contained in:
1 parent
27e63359d6
commit
6085a99dd8
14 files changed
+454
-192
No files matched your search
+35
-12
@@ -1,12 +1,18 @@
|
||||
#!/usr/bin/env bash
|
||||
# Download the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT payload for
|
||||
# frank-large from HuggingFace into /home/dank/models/.../Q8_0-AGENT.
|
||||
# frank-medium from HuggingFace into
|
||||
# /home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT.
|
||||
#
|
||||
# Uses the token in ~/.cache/huggingface/token so the ~28 GB main file does not
|
||||
# hit the anonymous 10 GB/30 min rate limit. Run on Dankclaw.
|
||||
# Run on Dankserver (the box that serves it). The repo is public, so the
|
||||
# download works anonymously; a token in ~/.cache/huggingface/token is used
|
||||
# when present, which keeps the ~28 GB main file clear of the anonymous
|
||||
# rate limit at full speed.
|
||||
#
|
||||
# bash scripts/download-q8agent.sh
|
||||
# bash scripts/download-q8agent.sh --verify-only
|
||||
#
|
||||
# The script is resumable (curl -C -) and verifies every artifact's size
|
||||
# afterwards, so a truncated file fails loudly instead of load-crashing.
|
||||
set -euo pipefail
|
||||
|
||||
REPO="kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF"
|
||||
@@ -16,6 +22,12 @@ EXPECTED=(
|
||||
"mmproj-Qwen3.8-27B-BF16.gguf:931145888"
|
||||
"mtp-Qwen3.8-27B-Q4_0.gguf:1680271648"
|
||||
)
|
||||
# The model card's MTP prompt-cache fix ships in the same repo as a patch;
|
||||
# fetched for reference only. This fork's main already carries the
|
||||
# speculative boundary in the context checkpoint, so nothing applies it.
|
||||
EXTRA=(
|
||||
"patches/mtp-prompt-cache-fix.patch"
|
||||
)
|
||||
|
||||
cd -- "$(dirname -- "$0")/.."
|
||||
|
||||
@@ -25,30 +37,38 @@ main() {
|
||||
mkdir -p "$DIR"
|
||||
cd "$DIR"
|
||||
|
||||
local token
|
||||
token="$(cat "$HOME/.cache/huggingface/token")"
|
||||
[[ -n "$token" ]] || { echo "no HF token in $HOME/.cache/huggingface/token" >&2; exit 1; }
|
||||
local token="" auth=()
|
||||
if [[ -r "$HOME/.cache/huggingface/token" ]]; then
|
||||
token="$(cat "$HOME/.cache/huggingface/token")"
|
||||
auth=(-H "Authorization: Bearer $token")
|
||||
else
|
||||
echo "no HF token at $HOME/.cache/huggingface/token; downloading anonymously"
|
||||
fi
|
||||
|
||||
dl() {
|
||||
local f="$1"
|
||||
curl -fsSL -C - -o "$f" --retry 5 --retry-delay 3 --retry-max-time 3600 \
|
||||
-H "Authorization: Bearer $token" \
|
||||
mkdir -p "$(dirname "$f")"
|
||||
curl -fsSL -C - -o "$f" --retry 10 --retry-delay 5 --retry-max-time 10800 \
|
||||
"${auth[@]}" \
|
||||
"https://huggingface.co/$REPO/resolve/main/$f"
|
||||
echo " done: $f ($(du -h "$f" | cut -f1))"
|
||||
}
|
||||
|
||||
if [[ "$mode" == "download" ]]; then
|
||||
echo "downloading $REPO into $DIR"
|
||||
dl "Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf"
|
||||
dl "mmproj-Qwen3.8-27B-BF16.gguf"
|
||||
dl "mtp-Qwen3.8-27B-Q4_0.gguf"
|
||||
for entry in "${EXPECTED[@]}"; do
|
||||
dl "${entry%%:*}"
|
||||
done
|
||||
for f in "${EXTRA[@]}"; do
|
||||
dl "$f"
|
||||
done
|
||||
fi
|
||||
|
||||
echo "verifying sizes"
|
||||
local ok=1 f want act
|
||||
for entry in "${EXPECTED[@]}"; do
|
||||
f="${entry%%:*}"; want="${entry##*:}"
|
||||
act="$(stat -c%s "$f")"
|
||||
act="$(stat -c%s "$f" 2>/dev/null || echo 0)"
|
||||
if [[ "$act" -eq "$want" ]]; then
|
||||
echo " OK $f ($act bytes)"
|
||||
else
|
||||
@@ -56,6 +76,9 @@ main() {
|
||||
ok=0
|
||||
fi
|
||||
done
|
||||
if [[ -z "$token" ]]; then
|
||||
echo "note: no token was used; sizes above are the integrity check"
|
||||
fi
|
||||
[[ "$ok" -eq 1 ]] || exit 1
|
||||
echo "all files present and sized correctly"
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user