serverAppConfigs/serverAppConfigs-docs/LLM/frank-large-unsloth-qwen38-27b-q8xl.md

---

type: reference title: stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large) description: The active frank-large runbook — the stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, UD-Q8_K_XL) with the DFlash2 drafter (z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M) on the Dankclaw Strix Halo iGPU: the prebuilt runtime, the full-window/one-stream shape, the DFlash2 spec method, fp8 KV, the Qwen-Sharp template, the payload, the client contract, and the failure modes worth knowing. tags:

• llm
• llama.cpp
• unsloth
• dankclaw
• frank-large
• strix-halo
• gfx1151
• q8
• dflash2
• rocm
• fp8-kv timestamp: 2026-09-21

---

stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)

ACTIVE since 2026-09-21. It replaced the Kairic Edge runtime (ciru-ai/ROCmFPX, jcbtc/Qwen3.8-27B-IU4-Kairic-Edge — see frank-large-kairic-edge-qwen38-27b, the current rollback target), which had replaced the unsloth llama.cpp prebuilt serving unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (125B MoE — see frank-large-unsloth-qwen38-flash-next). Deployed as a root systemd unit (scripts/frank-large.service, User=dank), port 7770, on the same gfx1151 iGPU. The RTX 3090 it does not touch belongs to frank-small-gemma4-12b.

The model is unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL — a 27B dense Qwen3.8 at UD-Q8_K_XL (a high-precision 8-bit quant). It is a qwen3_5 hybrid: 64 layers total, 16 full attention + 48 Gated DeltaNet linear attention. The payload is two files: the 31.46 GiB main GGUF and the 1.14 GiB DFlash2 Q4_K_M drafter.

The runtime is the stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) at /home/dank/GitPackages/unsloth-llama-b11030/llama-server. It is a mainline llama.cpp build that supports the qwen3_5 architecture and the DFlash2 drafter (llama_model_qwen35, llama_model_dflash, common_speculative_impl_draft_dflash).

Why this swap

• The operator asked for the 27B model at one conversation, full context (262,144) with DFlash2 spec decode (2026-09-21).
• The stock llama.cpp prebuilt is the only stock-compatible runtime with qwen3_5 + DFlash2 support — no source build, no CK config.h, no PromptForge sidecars, no compatibility mode.
• The spec method is DFlash2 (a block-diffusion drafter) at depth 4. The drafter has a sliding window of 2048 (small draft KV, ~40 MiB).

Shape

One root unit, one process, one published port:

half	port	reachable
llama-server (OpenAI front-end + engine, single process)	7770	published as host 7770

Runtime

• Prebuilt: /home/dank/GitPackages/unsloth-llama-b11030/llama-server (mainline, ROCm gfx1151 prebuilt, version 0.4.1-dev, build 11030, commit 6ba30d05b). The build bundles its own ROCm runtime — no host SDK gate, no venv.
• GPU target: gfx1151 (Strix Halo).
• Spec-draft flags: --spec-draft-type-k, --spec-draft-type-v, --spec-draft-threads, --spec-draft-cpu-mask, --spec-draft-prio, --spec-draft-poll all present (confirmed via --help).

Spec method

• DFlash2 depth 4 (--spec-type draft-dflash --model-draft <drafter> --spec-draft-n-max 4 --spec-draft-n-min 0): the block-diffusion drafter proposes 4 tokens per step.
• Draft device: --spec-draft-device ROCm0 (the drafter runs on the iGPU).
• DFlash2 is a block-diffusion drafter (not a standalone LM). The drafter has a sliding window of 2048 (~40 MiB of draft KV).

Chat template

The serve script passes --chat-template-file pointing at dankserver/frank-medium/qwen-sharp-v22.4.0.jinja (the Qwen-Sharp v22.4.0 template, the same one frank-medium uses). Without this flag the binary falls back to the built-in peg-native template, which produces a ~73,600-token prompt for a ~250-token request (280x the expected size). With the sharp template wired in, the same request produces a 62–175-token prompt. The Chat format: peg-native log line is the format-type label, not the template indicator — the sharp template is active even when that line appears.

Memory budget

The iGPU addresses 114 GiB (the GTT aperture over the shared LPDDR, gtt_total in /sys/class/drm/card1/device/; rocm-smi's 2 GiB "dedicated" is not the budget). The 27B dense model (31.46 GiB GGUF) is mmap'd in with -ngl 999; budget at 262k ≈ 41–43 GiB (weights + drafter

• fp8 KV + buffers), leaving ~71–73 GiB of aperture headroom. The 8 GiB prompt-cache budget (CACHE_RAM=8192) is the qualified config. The model is a 27B hybrid (16 full attention + 48 Gated DeltaNet); its DeltaNet state is smaller than a primarily-DeltaNet model, so the fleet's 12.8 GiB DeltaNet CACHE_RAM floor does not apply.

KV budget: 16 full-attention layers × 2 (K+V) × 4 kv_heads × 256 head_dim × 1 byte (fp8) = 32,768 bytes/token. 262,144 tokens × 32,768 ≈ 8 GiB for attention KV. Plus 48 DeltaNet recurrent states (small).

Payload

scripts/download-frank-large.sh (resumable, each file verified against its declared byte count + sha256; --verify-only re-checks):

file	size	sha256
Qwen3.8-27B-UD-Q8_K_XL.gguf	31,457,991,680	af36ecb6…
Qwen3.8-27B-DFlash2-Q4_K_M.gguf	1,143,006,816	1a25c568…

Client contract

• Port: 7770 (the same port and alias the prior deployments served).
• Alias: frank-large.
• Context: 262,144 (the model's native window), 1 slot.
• KV: fp8 quantized (--cache-type-k fp8 --cache-type-v fp8).
• Sampling: greedy (--temp 0 --top-p 1.0 --top-k 0 --min-p 0.0) — the DFlash2 drafter needs greedy verification.
• Reasoning: enabled (--reasoning auto --reasoning-format deepseek --reasoning-budget -1), extracted to reasoning_content.
• Vision: text-only (the model has a vision projector — mmproj-BF16 is available — but this deployment serves text-only, matching the prior frank-large). The fleet's image path is frank-small-gemma4-12b either way.
• Reasoning effort: the Qwen3.8 template gates reasoning_effort to {xhigh, medium, low} — the harness folds its levels onto that set in models.yml (reasoningEffortMap).

Measured performance

(To be filled in after deploy.)

Failure modes worth knowing

• The model has mtp_num_hidden_layers: 1 but this deployment uses DFlash2 (not MTP) for spec decode. The MTP layer is unused.
• The model has vision (language_model_only: false, image_token_id 248056, mmproj-BF16 0.93 GB available) but this deployment serves text-only.
• DFlash2 needs greedy verification: the serve script pins --temp 0 --top-p 1.0 --top-k 0 --min-p 0.0. A non-greedy request will not be verified correctly by the drafter.

Rollback

frank-large-kairic-edge-qwen38-27b — the Kairic Edge payload and runtime are intact on disk.