frank-large: swap to stock llama.cpp serving Qwen3.8-27B UD-Q8_K_XL with DFlash2
Replace the Kairic Edge ROCmFPX build with the unsloth b11030 stock llama.cpp prebuilt (ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF: UD-Q8_K_XL (a 27B dense model, 31.5 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) on the gfx1151 iGPU. Serving shape: one 262,144-token conversation, q8_0 KV (the build lacks fp8), text-only, greedy sampling, Qwen-Sharp v22.4.0 template. - scripts/serve-frank-large.sh: stock llama.cpp + DFlash2 spec decode - scripts/frank-large.env: new model/drafter/runtime, 262144 ctx / 1 slot - scripts/download-frank-large.sh: per-file fetch, verify mode, idempotent - scripts/frank-large.service: updated description - scripts/deploy-remote.sh: updated pkill pattern + asset gates - tests/frank_large_unit.bats: new model/runtime/spec assertions - docs: new runbook (active), Kairic Edge runbook retired (rollback target) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
1 parent
43b8258061
commit
7689092db8
10 files changed
+414
-306
No files matched your search
@@ -147,8 +147,15 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
|
||||
Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
|
||||
rollback record)
|
||||
- [[frank-large-kairic-edge-qwen38-27b]] — the ACTIVE frank-large runbook
|
||||
since 2026-09-21 (root systemd unit, port 7770): the Kairic Edge
|
||||
- [[frank-large-unsloth-qwen38-27b-q8xl]] — the ACTIVE frank-large runbook
|
||||
since 2026-09-21 (root systemd unit, port 7770): stock llama.cpp
|
||||
(unsloth b11030, ROCm gfx1151) serving
|
||||
unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (a 27B dense model) with the z-lab
|
||||
DFlash2 Q4_K_M drafter (depth 4) on the gfx1151 iGPU — one 262,144-token
|
||||
conversation, q8_0 KV (the build lacks fp8), text-only, Qwen-Sharp v22.4.0
|
||||
template, the 114 GiB GTT memory model, client contract
|
||||
- [[frank-large-kairic-edge-qwen38-27b]] — RETIRED 2026-09-21 (replaced by
|
||||
[[frank-large-unsloth-qwen38-27b-q8xl]]; same-day swap): the Kairic Edge
|
||||
runtime (ciru-ai/ROCmFPX kairic-edge-qwen38-27b-v1.2, TheRock 7.15.0,
|
||||
Composable Kernel) serving jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (a 27B
|
||||
dense model, main GGUF + FFN/GDN/GDN-Output .pfs sidecars) on the gfx1151
|
||||
|
||||
Reference in new issue
Block a user