feat(frank-large): unsloth llama.cpp Qwen3.8-Flash-Next UD-IQ4_XS (200k/1 slot, MTP depth 2)

Replace the CIRU v4.4.1 Orca runtime with the unsloth llama.cpp prebuilt
(b11030-mix-5ff778e, ROCm gfx1151, bundles its own ROCm) serving
unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (87.3 GiB, three HF split parts
concatenated) on the Strix Halo iGPU. One conversation, 200,000 ctx,
PARALLEL_SLOTS=1, MTP depth 2 via the explicit -md shared-Q8_0 head
(sidecar auto-discovery does not search the mtp/ subfolder), text-only.
Fleet contract unchanged: port 7770, alias frank-large.

- serve-frank-large.sh: unsloth binary, -m/-md, --spec-type draft-mtp
  --spec-draft-n-max 2, --jinja, preserve_thinking; gates on binary,
  GGUF, MTP head; conditional --mmproj on ENABLE_VISION=1
- download-frank-large-unsloth.sh: resumable fetch of 3 parts + MTP head
  + mmproj, each verified against its LFS sha256 fetched live from the
  Hub tree API; concatenates parts, records/verifies SHA256SUMS
- deploy-remote.sh: LLM_LARGE_MODEL/LLM_LARGE_DRAFT paths, pkill pattern
  on the model path (not the binary name), wait_port_free gate
- tests: seams for 200k/1-slot, explicit -md, text-only, handler gates,
  sweep safety
This commit is contained in:
Frank committed 2026-09-21 12:30:30 -07:00
1 parent e5a1f8e767
commit b149d02187
6 files changed
+272 -157

No files matched your search

+14 -17
View File
@@ -1028,22 +1028,20 @@ deploy_anubis() {
wait_container_healthy anubis 60 || return 1
}
# frank-large is the CIRU v4.4.1 runtime (QSA window hotfix) serving
# jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca on the Strix Halo iGPU, as a ROOT
# unit since the 2026-09-19 cutover: the runtime is a host build against
# the private ROCm 10 SDK, so there is no container image to install and no
# generated unit to dodge. relink_unit points
# frank-large is the unsloth llama.cpp prebuilt (b11030-mix-5ff778e, ROCm
# gfx1151) serving unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS on the Strix
# Halo iGPU, as a ROOT unit since the 2026-09-19 cutover: the runtime is a
# prebuilt tarball with its own bundled ROCm, so there is no container
# image to install and no generated unit to dodge. relink_unit points
# /etc/systemd/system/frank-large.service at the repo file and boot
# persistence is that unit's [Install] section.
#
# It replaces the halogen-flash-server quadlet, which served
# peonist-ai/halogen-qwen3.8-flash-next on this same iGPU since 2026-09-16.
# The quadlet is retired HERE (the unit file moves into the home trash,
# never `rm`), and the port gate is the same as the 2026-09-16 cutover:
# nothing starts until 7770 is released.
LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/Qwen3.8-Flash-CIRU-STRIX-Orca/Qwen3.8-Flash-CIRU-STRIX-Orca.gguf}"
LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/Qwen3.8-Flash-CIRU-STRIX-Orca/mtp/Qwen3.8-Flash-CIRU-STRIX-Orca-MTP-Q8_0.gguf}"
LLM_LARGE_PLE="${LLM_LARGE_PLE:-/home/dank/models/Qwen3.8-Flash-CIRU-STRIX-Orca/ple/ple.payload.bin}"
# It replaces the CIRU v4.4.1 Orca runtime (jcbtc/Qwen3.8-Flash-CIRU-
# STRIX-Orca), which had replaced the halogen-flash-server quadlet. The
# port gate is the same as every cutover since 2026-09-16: nothing starts
# until 7770 is released.
LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/Qwen3.8-Flash-Next-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS.gguf}"
LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/Qwen3.8-Flash-Next-UD-IQ4_XS/mtp/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf}"
retire_halogen_large_quadlet() {
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
@@ -1080,10 +1078,10 @@ deploy_llm_large() {
# missing never becomes healthy, and a green deploy would leave the
# endpoint dark with nothing in the deploy log to say why.
local missing=0 f
for f in "$LLM_LARGE_MODEL" "$LLM_LARGE_DRAFT" "$LLM_LARGE_PLE"; do
for f in "$LLM_LARGE_MODEL" "$LLM_LARGE_DRAFT"; do
if [[ ! -f "$f" ]]; then
echo "ERROR: payload absent at $f" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-orca.sh" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-unsloth.sh" >&2
missing=1
fi
done
@@ -1098,8 +1096,7 @@ deploy_llm_large() {
# The pattern never appears in the deploying shell's own command line, so
# -f is safe here (it is not in `bash deploy-remote.sh`).
run_privileged systemctl stop frank-large.service 2>/dev/null || true
pkill -f 'Qwen3.8-Flash-CIRU-STRIX-Orca' 2>/dev/null || true
sleep 2
pkill -f 'Qwen3.8-Flash-Next-UD-IQ4_XS' 2>/dev/null || true
# Nothing may start until the predecessor has released the port. On
# 2026-09-16 the cutover failed exactly here: the old server logged
# "Deactivated successfully", the next engine tried to bind 7770 a second