frank-large: swap to Kairic Edge 27B IU4 (W4A4) on Strix Halo

Replace the unsloth llama.cpp prebuilt (b11030, Qwen3.8-Flash-Next
UD-IQ4_XS, 125B MoE) with the Kairic Edge runtime (ciru-ai/ROCmFPX,
branch kairic-edge-qwen38-27b-v1.2, commit 205a3e5) serving
jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (27B dense, IU4 W4A4).

- Runtime: built from source (scripts/build-kairic-edge-gfx1151.sh),
  TheRock 7.15.0a20260718 toolchain, CK fdf4bb7f + gfx1151-iu4 patch.
- Shape: one stream, full 262144 context (the model's native window),
  1 slot, 8 GiB prompt-cache budget.
- Spec method: native MTP depth 4 + n-gram drafting (24/64/64). DFlash
  is NOT available for this model (no DRAFT_DFLASH spec type).
- Compatibility mode: the harness needs sampling, tools and a reasoning
  budget, which fast-greedy rejects. COMPATIBILITY_MODE=1 enables them.
- Reasoning: enabled (auto + deepseek format, thoughts land in
  reasoning_content).
- Payload: 4 files (15.5 GiB GGUF + 7.99 GiB FFN + 1.88 GiB GDN + 0.70
  GiB GDN-Output), sha256-verified (scripts/download-frank-large-kairic.sh).
This commit is contained in:
Frank
2026-09-21 18:56:35 -07:00
parent 2f7aadab88
commit b2a74004df
7 changed files with 410 additions and 155 deletions
+13 -12
View File
@@ -1072,20 +1072,21 @@ deploy_anubis() {
wait_container_healthy anubis 60 || return 1
}
# frank-large is the unsloth llama.cpp prebuilt (b11030-mix-5ff778e, ROCm
# gfx1151) serving unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS on the Strix
# Halo iGPU, as a ROOT unit since the 2026-09-19 cutover: the runtime is a
# prebuilt tarball with its own bundled ROCm, so there is no container
# image to install and no generated unit to dodge. relink_unit points
# frank-large is the Kairic Edge runtime (ciru-ai/ROCmFPX, branch
# kairic-edge-qwen38-27b-v1.2, commit 205a3e5) serving
# jcbtc/Qwen3.8-27B-IU4-Kairic-Edge on the Strix Halo iGPU, as a ROOT
# unit since the 2026-09-19 cutover: the runtime is built from source
# (scripts/build-kairic-edge-gfx1151.sh), so there is no container image
# to install and no generated unit to dodge. relink_unit points
# /etc/systemd/system/frank-large.service at the repo file and boot
# persistence is that unit's [Install] section.
#
# It replaces the CIRU v4.4.1 Orca runtime (jcbtc/Qwen3.8-Flash-CIRU-
# STRIX-Orca), which had replaced the halogen-flash-server quadlet. The
# port gate is the same as every cutover since 2026-09-16: nothing starts
# It replaces the unsloth llama.cpp prebuilt (b11030-mix-5ff778e, ROCm
# gfx1151), which had replaced the CIRU v4.4.1 Orca runtime. The port
# gate is the same as every cutover since 2026-09-16: nothing starts
# until 7770 is released.
LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/Qwen3.8-Flash-Next-UD-IQ4_XS/UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf}"
LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/Qwen3.8-Flash-Next-UD-IQ4_XS/mtp/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf}"
LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/Qwen3.8-27B-IU4-Kairic-Edge/Qwen3.8-27B-IU4-Kairic-Edge.gguf}"
LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/Qwen3.8-27B-IU4-Kairic-Edge/Qwen3.8-27B-Kairic-IU4-FFN.pfs}"
retire_halogen_large_quadlet() {
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
@@ -1125,7 +1126,7 @@ deploy_llm_large() {
for f in "$LLM_LARGE_MODEL" "$LLM_LARGE_DRAFT"; do
if [[ ! -f "$f" ]]; then
echo "ERROR: payload absent at $f" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-unsloth.sh" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-kairic.sh" >&2
missing=1
fi
done
@@ -1140,7 +1141,7 @@ deploy_llm_large() {
# The pattern never appears in the deploying shell's own command line, so
# -f is safe here (it is not in `bash deploy-remote.sh`).
run_privileged systemctl stop frank-large.service 2>/dev/null || true
pkill -f 'Qwen3.8-Flash-Next-UD-IQ4_XS' 2>/dev/null || true
pkill -f 'Qwen3.8-27B-IU4-Kairic-Edge' 2>/dev/null || true
# Nothing may start until the predecessor has released the port. On
# 2026-09-16 the cutover failed exactly here: the old server logged
# "Deactivated successfully", the next engine tried to bind 7770 a second