frank-large: swap to stock llama.cpp serving Qwen3.8-27B UD-Q8_K_XL with DFlash2

Replace the Kairic Edge ROCmFPX build with the unsloth b11030 stock
llama.cpp prebuilt (ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:
UD-Q8_K_XL (a 27B dense model, 31.5 GiB) with the z-lab DFlash2 Q4_K_M
drafter (depth 4) on the gfx1151 iGPU.

Serving shape: one 262,144-token conversation, q8_0 KV (the build lacks
fp8), text-only, greedy sampling, Qwen-Sharp v22.4.0 template.

- scripts/serve-frank-large.sh: stock llama.cpp + DFlash2 spec decode
- scripts/frank-large.env: new model/drafter/runtime, 262144 ctx / 1 slot
- scripts/download-frank-large.sh: per-file fetch, verify mode, idempotent
- scripts/frank-large.service: updated description
- scripts/deploy-remote.sh: updated pkill pattern + asset gates
- tests/frank_large_unit.bats: new model/runtime/spec assertions
- docs: new runbook (active), Kairic Edge runbook retired (rollback target)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
FrankandClaude committed 2026-09-21 21:50:36 -07:00
1 parent 43b8258061
commit 7689092db8
10 files changed
+414 -306

No files matched your search

+23 -15
View File
@@ -4,11 +4,11 @@ Documentation for the LLM inference fleet on the two hosts (2026-09-21):
**vllm-radiance on Dankserver** serving frank-medium as ONE tensor-parallel
process across both Radeon AI PRO R9700 cards (just1moremodel
Qwen3.8-27B-Uncensored-MXFP4 + the z-lab DFlash2 drafter, bf16 KV, TP=2 over the
R4D one-shot allreduce), and TWO endpoints on Dankclaw — the Kairic Edge
runtime (frank-large, ciru-ai/ROCmFPX v1.2, jcbtc Qwen3.8-27B-IU4-Kairic-Edge
IU4 W4A4 on the gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small,
HauhauCS Gemma4-12B QAT Q4_K_M with MTP and vision on the RTX 3090, port
7771).
R4D one-shot allreduce), and TWO endpoints on Dankclaw — stock llama.cpp
(unsloth b11030, ROCm gfx1151; frank-large, unsloth Qwen3.8-27B UD-Q8_K_XL +
the z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU, port 7770) and mainline
llama.cpp (frank-small, HauhauCS Gemma4-12B QAT Q4_K_M with MTP and vision on
the RTX 3090, port 7771).
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -32,10 +32,10 @@ ONE process on both cards, 262144 ctx, up to 8 seqs]
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
E2[RTX 3090 (PCIe x4 dock) +
gfx1151 iGPU 114 GiB GTT]
E[Kairic Edge (ciru-ai/ROCmFPX v1.2, gfx1151)
E[stock llama.cpp (unsloth b11030, gfx1151)
frank-large, port 7770]
F[Qwen3.8-27B-IU4-Kairic-Edge (IU4 W4A4)
262144 ctx / 1 slot, MTP depth 4 + n-gram]
F[Qwen3.8-27B UD-Q8_K_XL + DFlash2 Q4_K_M drafter
262144 ctx / 1 slot, DFlash2 depth 4, q8_0 KV]
G[mainline llama.cpp 37b53fd
frank-small, port 7771]
H[Gemma4-12B QAT Q4_K_M
@@ -96,13 +96,21 @@ frank-small, port 7771]
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-large-kairic-edge-qwen38-27b]] — ACTIVE frank-large runbook
since 2026-09-21 (root systemd unit, port 7770): the Kairic Edge runtime
(ciru-ai/ROCmFPX v1.2, built from source) serving
jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (IU4 W4A4, 27B dense) on the gfx1151
iGPU — one full 262,144-token conversation, MTP depth 4 + n-gram drafting,
the three PromptForge sidecars, compatibility mode, the 114 GiB GTT memory
model, client contract, troubleshooting
- [[frank-large-unsloth-qwen38-27b-q8xl]] — ACTIVE frank-large runbook
since 2026-09-21 (root systemd unit, port 7770): stock llama.cpp
(unsloth b11030, ROCm gfx1151) serving
unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense) with the z-lab DFlash2
Q4_K_M drafter (depth 4) on the gfx1151 iGPU — one full 262,144-token
conversation, q8_0 KV (the build lacks fp8), text-only, Qwen-Sharp
v22.4.0 template, the 114 GiB GTT memory model, client contract,
troubleshooting
- [[frank-large-kairic-edge-qwen38-27b]] — RETIRED 2026-09-21 (same-day
swap, replaced by [[frank-large-unsloth-qwen38-27b-q8xl]]; rollback
target): the Kairic Edge runtime (ciru-ai/ROCmFPX v1.2, built from
source) serving jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (IU4 W4A4, 27B dense)
on the gfx1151 iGPU — one full 262,144-token conversation, MTP depth 4 +
n-gram drafting, the three PromptForge sidecars, compatibility mode, the
114 GiB GTT memory model, client contract, troubleshooting
- [[frank-large-unsloth-qwen38-flash-next]] — RETIRED 2026-09-21 (same-day
swap, replaced by [[frank-large-kairic-edge-qwen38-27b]]; rollback target):
the unsloth llama.cpp prebuilt (b11030-mix-5ff778e, ROCm gfx1151) serving