frank-medium: serve the new R9700s, retire frank-small

The RTX 3090 left Dankserver; two AMD Radeon AI PRO R9700 cards (Navi 48,
gfx1201, 32,624 MiB each) replaced it. FreeToken/CUDA cannot run there at all,
so the `frank-small` service is retired and `frank-medium` takes the endpoint.

New service: charlie12345/ROCmFPX llama.cpp (built for gfx1201 against the
fork's self-contained ROCm 7.14 toolchain, so the host's stale /opt/rocm-6.3.3
never participates) serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF --
a Q8_0_ROCMFPX_AGENT (ftype 115) checkpoint that stock llama.cpp cannot load.
Placeholder config: --split-mode tensor across both cards, 131072 context,
1 slot, f16 KV, MTP draft head on ROCm0.

deploy-remote.sh grows retire_root_llm_small(): the CUDA-era unit is stopped,
disabled and moved to ~/.AGENTTRASH/20260915-frank-small-retire/host-units
before frank-medium starts, because `systemctl disable` has been observed to
delete the unit file itself on this host (2026-08-26 incident).

The retired frank-small repo files (env, unit, serve script) move to
scripts/.AGENTTRASH/, and the planner, dispatcher, service allow-list and the
deployment test seams now name frank-medium.
This commit is contained in:
Frank committed 2026-09-15 18:53:20 -07:00
1 parent 27e63359d6
commit 6085a99dd8
14 files changed
+454 -192

No files matched your search

+35 -12
View File
@@ -1,12 +1,18 @@
#!/usr/bin/env bash
# Download the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT payload for
# frank-large from HuggingFace into /home/dank/models/.../Q8_0-AGENT.
# frank-medium from HuggingFace into
# /home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT.
#
# Uses the token in ~/.cache/huggingface/token so the ~28 GB main file does not
# hit the anonymous 10 GB/30 min rate limit. Run on Dankclaw.
# Run on Dankserver (the box that serves it). The repo is public, so the
# download works anonymously; a token in ~/.cache/huggingface/token is used
# when present, which keeps the ~28 GB main file clear of the anonymous
# rate limit at full speed.
#
# bash scripts/download-q8agent.sh
# bash scripts/download-q8agent.sh --verify-only
#
# The script is resumable (curl -C -) and verifies every artifact's size
# afterwards, so a truncated file fails loudly instead of load-crashing.
set -euo pipefail
REPO="kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF"
@@ -16,6 +22,12 @@ EXPECTED=(
"mmproj-Qwen3.8-27B-BF16.gguf:931145888"
"mtp-Qwen3.8-27B-Q4_0.gguf:1680271648"
)
# The model card's MTP prompt-cache fix ships in the same repo as a patch;
# fetched for reference only. This fork's main already carries the
# speculative boundary in the context checkpoint, so nothing applies it.
EXTRA=(
"patches/mtp-prompt-cache-fix.patch"
)
cd -- "$(dirname -- "$0")/.."
@@ -25,30 +37,38 @@ main() {
mkdir -p "$DIR"
cd "$DIR"
local token
token="$(cat "$HOME/.cache/huggingface/token")"
[[ -n "$token" ]] || { echo "no HF token in $HOME/.cache/huggingface/token" >&2; exit 1; }
local token="" auth=()
if [[ -r "$HOME/.cache/huggingface/token" ]]; then
token="$(cat "$HOME/.cache/huggingface/token")"
auth=(-H "Authorization: Bearer $token")
else
echo "no HF token at $HOME/.cache/huggingface/token; downloading anonymously"
fi
dl() {
local f="$1"
curl -fsSL -C - -o "$f" --retry 5 --retry-delay 3 --retry-max-time 3600 \
-H "Authorization: Bearer $token" \
mkdir -p "$(dirname "$f")"
curl -fsSL -C - -o "$f" --retry 10 --retry-delay 5 --retry-max-time 10800 \
"${auth[@]}" \
"https://huggingface.co/$REPO/resolve/main/$f"
echo " done: $f ($(du -h "$f" | cut -f1))"
}
if [[ "$mode" == "download" ]]; then
echo "downloading $REPO into $DIR"
dl "Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf"
dl "mmproj-Qwen3.8-27B-BF16.gguf"
dl "mtp-Qwen3.8-27B-Q4_0.gguf"
for entry in "${EXPECTED[@]}"; do
dl "${entry%%:*}"
done
for f in "${EXTRA[@]}"; do
dl "$f"
done
fi
echo "verifying sizes"
local ok=1 f want act
for entry in "${EXPECTED[@]}"; do
f="${entry%%:*}"; want="${entry##*:}"
act="$(stat -c%s "$f")"
act="$(stat -c%s "$f" 2>/dev/null || echo 0)"
if [[ "$act" -eq "$want" ]]; then
echo " OK $f ($act bytes)"
else
@@ -56,6 +76,9 @@ main() {
ok=0
fi
done
if [[ -z "$token" ]]; then
echo "note: no token was used; sizes above are the integrity check"
fi
[[ "$ok" -eq 1 ]] || exit 1
echo "all files present and sized correctly"
}