frank-large: swap to stock llama.cpp serving Qwen3.8-27B UD-Q8_K_XL with DFlash2

Replace the Kairic Edge ROCmFPX build with the unsloth b11030 stock
llama.cpp prebuilt (ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:
UD-Q8_K_XL (a 27B dense model, 31.5 GiB) with the z-lab DFlash2 Q4_K_M
drafter (depth 4) on the gfx1151 iGPU.

Serving shape: one 262,144-token conversation, q8_0 KV (the build lacks
fp8), text-only, greedy sampling, Qwen-Sharp v22.4.0 template.

- scripts/serve-frank-large.sh: stock llama.cpp + DFlash2 spec decode
- scripts/frank-large.env: new model/drafter/runtime, 262144 ctx / 1 slot
- scripts/download-frank-large.sh: per-file fetch, verify mode, idempotent
- scripts/frank-large.service: updated description
- scripts/deploy-remote.sh: updated pkill pattern + asset gates
- tests/frank_large_unit.bats: new model/runtime/spec assertions
- docs: new runbook (active), Kairic Edge runbook retired (rollback target)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
FrankandClaude committed 2026-09-21 21:50:36 -07:00
1 parent 43b8258061
commit 7689092db8
10 files changed
+414 -306

No files matched your search

+14 -20
View File
@@ -1,6 +1,6 @@
# frank-large — Kairic Edge runtime (ciru-ai/ROCmFPX, branch
# kairic-edge-qwen38-27b-v1.2, commit 205a3e5) serving
# jcbtc/Qwen3.8-27B-IU4-Kairic-Edge on the Strix Halo iGPU (gfx1151).
# frank-large — stock llama.cpp (mainline, the unsloth b11030 prebuilt)
# serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL on the Strix Halo iGPU
# (gfx1151).
#
# These are the tunables read by scripts/serve-frank-large.sh. The flag
# list is owned by the serve script; this file only sets fleet values.
@@ -9,15 +9,14 @@
# runtime reads those bare (the 2026-08-13 TEMP incident). HOST is the one
# exception because the serve script itself consumes it for --host.
# The assembled payload (scripts/download-frank-large-kairic.sh): the
# serve script refuses to start without the main GGUF and the three
# sidecars (FFN, GDN, GDN-Output).
MODEL_DIR=/home/dank/models/Qwen3.8-27B-IU4-Kairic-Edge
# The payload (scripts/download-frank-large.sh): the serve script refuses to
# start without the main GGUF and the DFlash2 drafter.
MODEL_DIR=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL
# The Kairic Edge runtime (scripts/build-kairic-edge-gfx1151.sh).
RUNTIME_DIR=/home/dank/GitPackages/ROCmFPX-kairic/build-kairic
# The stock llama.cpp runtime (the unsloth b11030 prebuilt).
RUNTIME_DIR=/home/dank/GitPackages/unsloth-llama-b11030
# Fleet endpoint: the same port and alias the unsloth predecessor served, so
# Fleet endpoint: the same port and alias the prior deployments served, so
# no client, probe or models.yml entry changes address.
HOST=0.0.0.0
PORT=7770
@@ -26,17 +25,12 @@ PORT=7770
CONTEXT_SIZE=262144
PARALLEL_SLOTS=1
# Native MTP depth 4 (the model's MTP layers) + n-gram drafting (24/64/64).
# DFlash is NOT available for this model (no DRAFT_DFLASH spec type in the
# Kairic Edge runtime).
# DFlash2 spec decode (a block-diffusion drafter) at depth 4. The drafter
# has a sliding window of 2048 (small draft KV, ~40 MiB).
SPEC_DRAFT_N_MAX=4
# The harness needs sampling, tools and a reasoning budget, which the
# fast-greedy path rejects. COMPATIBILITY_MODE=1 enables them (9.7% slower
# per the build guide).
COMPATIBILITY_MODE=1
# Prompt-cache budget: 8 GiB (the Kairic Edge qualified config). The model
# is 27B dense (not DeltaNet), so the fleet's 12.8 GiB DeltaNet floor does
# Prompt-cache budget: 8 GiB. The model is a 27B hybrid (16 full attention
# + 48 Gated DeltaNet linear attention); its DeltaNet state is smaller than
# a primarily-DeltaNet model, so the fleet's 12.8 GiB DeltaNet floor does
# not apply.
CACHE_RAM=8192