frank-large: swap to stock llama.cpp serving Qwen3.8-27B UD-Q8_K_XL with DFlash2

Replace the Kairic Edge ROCmFPX build with the unsloth b11030 stock
llama.cpp prebuilt (ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:
UD-Q8_K_XL (a 27B dense model, 31.5 GiB) with the z-lab DFlash2 Q4_K_M
drafter (depth 4) on the gfx1151 iGPU.

Serving shape: one 262,144-token conversation, q8_0 KV (the build lacks
fp8), text-only, greedy sampling, Qwen-Sharp v22.4.0 template.

- scripts/serve-frank-large.sh: stock llama.cpp + DFlash2 spec decode
- scripts/frank-large.env: new model/drafter/runtime, 262144 ctx / 1 slot
- scripts/download-frank-large.sh: per-file fetch, verify mode, idempotent
- scripts/frank-large.service: updated description
- scripts/deploy-remote.sh: updated pkill pattern + asset gates
- tests/frank_large_unit.bats: new model/runtime/spec assertions
- docs: new runbook (active), Kairic Edge runbook retired (rollback target)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Frank
2026-09-21 21:50:36 -07:00
co-authored by Claude
parent 43b8258061
commit 7689092db8
10 changed files with 414 additions and 306 deletions
+15 -16
View File
@@ -1072,21 +1072,20 @@ deploy_anubis() {
wait_container_healthy anubis 60 || return 1
}
# frank-large is the Kairic Edge runtime (ciru-ai/ROCmFPX, branch
# kairic-edge-qwen38-27b-v1.2, commit 205a3e5) serving
# jcbtc/Qwen3.8-27B-IU4-Kairic-Edge on the Strix Halo iGPU, as a ROOT
# unit since the 2026-09-19 cutover: the runtime is built from source
# (scripts/build-kairic-edge-gfx1151.sh), so there is no container image
# to install and no generated unit to dodge. relink_unit points
# /etc/systemd/system/frank-large.service at the repo file and boot
# persistence is that unit's [Install] section.
# frank-large is the stock llama.cpp prebuilt (unsloth b11030, ROCm
# gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the DFlash2
# drafter (z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M) on the Strix Halo iGPU,
# as a ROOT unit since the 2026-09-19 cutover: the runtime is a prebuilt
# binary, so there is no container image to install and no generated unit
# to dodge. relink_unit points /etc/systemd/system/frank-large.service at
# the repo file and boot persistence is that unit's [Install] section.
#
# It replaces the unsloth llama.cpp prebuilt (b11030-mix-5ff778e, ROCm
# gfx1151), which had replaced the CIRU v4.4.1 Orca runtime. The port
# gate is the same as every cutover since 2026-09-16: nothing starts
# until 7770 is released.
LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/Qwen3.8-27B-IU4-Kairic-Edge/Qwen3.8-27B-IU4-Kairic-Edge.gguf}"
LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/Qwen3.8-27B-IU4-Kairic-Edge/Qwen3.8-27B-Kairic-IU4-FFN.pfs}"
# It replaces the Kairic Edge runtime (ciru-ai/ROCmFPX), which had
# replaced the unsloth llama.cpp prebuilt serving
# unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS. The port gate is the same as
# every cutover since 2026-09-16: nothing starts until 7770 is released.
LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-UD-Q8_K_XL.gguf}"
LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf}"
retire_halogen_large_quadlet() {
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
@@ -1126,7 +1125,7 @@ deploy_llm_large() {
for f in "$LLM_LARGE_MODEL" "$LLM_LARGE_DRAFT"; do
if [[ ! -f "$f" ]]; then
echo "ERROR: payload absent at $f" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-kairic.sh" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large.sh" >&2
missing=1
fi
done
@@ -1141,7 +1140,7 @@ deploy_llm_large() {
# The pattern never appears in the deploying shell's own command line, so
# -f is safe here (it is not in `bash deploy-remote.sh`).
run_privileged systemctl stop frank-large.service 2>/dev/null || true
pkill -f 'Qwen3.8-27B-IU4-Kairic-Edge' 2>/dev/null || true
pkill -f 'Qwen3.8-27B-UD-Q8_K_XL' 2>/dev/null || true
# Nothing may start until the predecessor has released the port. On
# 2026-09-16 the cutover failed exactly here: the old server logged
# "Deactivated successfully", the next engine tried to bind 7770 a second