frank-large: swap to stock llama.cpp serving Qwen3.8-27B UD-Q8_K_XL with DFlash2
Replace the Kairic Edge ROCmFPX build with the unsloth b11030 stock llama.cpp prebuilt (ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF: UD-Q8_K_XL (a 27B dense model, 31.5 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) on the gfx1151 iGPU. Serving shape: one 262,144-token conversation, q8_0 KV (the build lacks fp8), text-only, greedy sampling, Qwen-Sharp v22.4.0 template. - scripts/serve-frank-large.sh: stock llama.cpp + DFlash2 spec decode - scripts/frank-large.env: new model/drafter/runtime, 262144 ctx / 1 slot - scripts/download-frank-large.sh: per-file fetch, verify mode, idempotent - scripts/frank-large.service: updated description - scripts/deploy-remote.sh: updated pkill pattern + asset gates - tests/frank_large_unit.bats: new model/runtime/spec assertions - docs: new runbook (active), Kairic Edge runbook retired (rollback target) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
1 parent
43b8258061
commit
7689092db8
10 files changed
+414
-306
No files matched your search
@@ -4,11 +4,11 @@ Documentation for the LLM inference fleet on the two hosts (2026-09-21):
|
||||
**vllm-radiance on Dankserver** serving frank-medium as ONE tensor-parallel
|
||||
process across both Radeon AI PRO R9700 cards (just1moremodel
|
||||
Qwen3.8-27B-Uncensored-MXFP4 + the z-lab DFlash2 drafter, bf16 KV, TP=2 over the
|
||||
R4D one-shot allreduce), and TWO endpoints on Dankclaw — the Kairic Edge
|
||||
runtime (frank-large, ciru-ai/ROCmFPX v1.2, jcbtc Qwen3.8-27B-IU4-Kairic-Edge
|
||||
IU4 W4A4 on the gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small,
|
||||
HauhauCS Gemma4-12B QAT Q4_K_M with MTP and vision on the RTX 3090, port
|
||||
7771).
|
||||
R4D one-shot allreduce), and TWO endpoints on Dankclaw — stock llama.cpp
|
||||
(unsloth b11030, ROCm gfx1151; frank-large, unsloth Qwen3.8-27B UD-Q8_K_XL +
|
||||
the z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU, port 7770) and mainline
|
||||
llama.cpp (frank-small, HauhauCS Gemma4-12B QAT Q4_K_M with MTP and vision on
|
||||
the RTX 3090, port 7771).
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -32,10 +32,10 @@ ONE process on both cards, 262144 ctx, up to 8 seqs]
|
||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||
E2[RTX 3090 (PCIe x4 dock) +
|
||||
gfx1151 iGPU 114 GiB GTT]
|
||||
E[Kairic Edge (ciru-ai/ROCmFPX v1.2, gfx1151)
|
||||
E[stock llama.cpp (unsloth b11030, gfx1151)
|
||||
frank-large, port 7770]
|
||||
F[Qwen3.8-27B-IU4-Kairic-Edge (IU4 W4A4)
|
||||
262144 ctx / 1 slot, MTP depth 4 + n-gram]
|
||||
F[Qwen3.8-27B UD-Q8_K_XL + DFlash2 Q4_K_M drafter
|
||||
262144 ctx / 1 slot, DFlash2 depth 4, q8_0 KV]
|
||||
G[mainline llama.cpp 37b53fd
|
||||
frank-small, port 7771]
|
||||
H[Gemma4-12B QAT Q4_K_M
|
||||
@@ -96,13 +96,21 @@ frank-small, port 7771]
|
||||
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
|
||||
- [[deployment]] — deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[frank-large-kairic-edge-qwen38-27b]] — ACTIVE frank-large runbook
|
||||
since 2026-09-21 (root systemd unit, port 7770): the Kairic Edge runtime
|
||||
(ciru-ai/ROCmFPX v1.2, built from source) serving
|
||||
jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (IU4 W4A4, 27B dense) on the gfx1151
|
||||
iGPU — one full 262,144-token conversation, MTP depth 4 + n-gram drafting,
|
||||
the three PromptForge sidecars, compatibility mode, the 114 GiB GTT memory
|
||||
model, client contract, troubleshooting
|
||||
- [[frank-large-unsloth-qwen38-27b-q8xl]] — ACTIVE frank-large runbook
|
||||
since 2026-09-21 (root systemd unit, port 7770): stock llama.cpp
|
||||
(unsloth b11030, ROCm gfx1151) serving
|
||||
unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense) with the z-lab DFlash2
|
||||
Q4_K_M drafter (depth 4) on the gfx1151 iGPU — one full 262,144-token
|
||||
conversation, q8_0 KV (the build lacks fp8), text-only, Qwen-Sharp
|
||||
v22.4.0 template, the 114 GiB GTT memory model, client contract,
|
||||
troubleshooting
|
||||
- [[frank-large-kairic-edge-qwen38-27b]] — RETIRED 2026-09-21 (same-day
|
||||
swap, replaced by [[frank-large-unsloth-qwen38-27b-q8xl]]; rollback
|
||||
target): the Kairic Edge runtime (ciru-ai/ROCmFPX v1.2, built from
|
||||
source) serving jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (IU4 W4A4, 27B dense)
|
||||
on the gfx1151 iGPU — one full 262,144-token conversation, MTP depth 4 +
|
||||
n-gram drafting, the three PromptForge sidecars, compatibility mode, the
|
||||
114 GiB GTT memory model, client contract, troubleshooting
|
||||
- [[frank-large-unsloth-qwen38-flash-next]] — RETIRED 2026-09-21 (same-day
|
||||
swap, replaced by [[frank-large-kairic-edge-qwen38-27b]]; rollback target):
|
||||
the unsloth llama.cpp prebuilt (b11030-mix-5ff778e, ROCm gfx1151) serving
|
||||
|
||||
Reference in new issue
Block a user