--- type: reference title: stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large) description: The active frank-large runbook — the stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, UD-Q8_K_XL) with the DFlash2 drafter (z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M) on the Dankclaw Strix Halo iGPU: the prebuilt runtime, the full-window/one-stream shape, the DFlash2 spec method, fp8 KV, the Qwen-Sharp template, the payload, the client contract, and the failure modes worth knowing. tags: - llm - llama.cpp - unsloth - dankclaw - frank-large - strix-halo - gfx1151 - q8 - dflash2 - rocm - fp8-kv timestamp: 2026-09-21 --- # stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large) > **ACTIVE since 2026-09-21.** It replaced the Kairic Edge runtime > (ciru-ai/ROCmFPX, jcbtc/Qwen3.8-27B-IU4-Kairic-Edge — see > [[frank-large-kairic-edge-qwen38-27b]], the current rollback target), > which had replaced the unsloth llama.cpp prebuilt serving > unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (125B MoE — see > [[frank-large-unsloth-qwen38-flash-next]]). Deployed as a **root > systemd unit** (`scripts/frank-large.service`, `User=dank`), port 7770, > on the same gfx1151 iGPU. The RTX 3090 it does not touch belongs to > [[frank-small-gemma4-12b]]. The model is `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL` — a **27B dense** Qwen3.8 at **UD-Q8_K_XL** (a high-precision 8-bit quant). It is a qwen3_5 hybrid: 64 layers total, 16 full attention + 48 Gated DeltaNet linear attention. The payload is two files: the 31.46 GiB main GGUF and the 1.14 GiB DFlash2 Q4_K_M drafter. The runtime is the **stock llama.cpp** prebuilt (unsloth b11030, ROCm gfx1151) at `/home/dank/GitPackages/unsloth-llama-b11030/llama-server`. It is a mainline llama.cpp build that supports the qwen3_5 architecture and the DFlash2 drafter (llama_model_qwen35, llama_model_dflash, common_speculative_impl_draft_dflash). ## Why this swap - The operator asked for the 27B model at **one conversation, full context (262,144)** with DFlash2 spec decode (2026-09-21). - The stock llama.cpp prebuilt is the only stock-compatible runtime with qwen3_5 + DFlash2 support — no source build, no CK `config.h`, no PromptForge sidecars, no compatibility mode. - The spec method is DFlash2 (a block-diffusion drafter) at depth 4. The drafter has a sliding window of 2048 (small draft KV, ~40 MiB). ## Shape One root unit, one process, one published port: | half | port | reachable | |---|---|---| | llama-server (OpenAI front-end + engine, single process) | 7770 | published as host **7770** | ## Runtime - **Prebuilt**: `/home/dank/GitPackages/unsloth-llama-b11030/llama-server` (mainline, ROCm gfx1151 prebuilt, version 0.4.1-dev, build 11030, commit 6ba30d05b). The build bundles its own ROCm runtime — no host SDK gate, no venv. - **GPU target**: `gfx1151` (Strix Halo). - **Spec-draft flags**: `--spec-draft-type-k`, `--spec-draft-type-v`, `--spec-draft-threads`, `--spec-draft-cpu-mask`, `--spec-draft-prio`, `--spec-draft-poll` all present (confirmed via `--help`). ## Spec method - **DFlash2 depth 4** (`--spec-type draft-dflash --model-draft --spec-draft-n-max 4 --spec-draft-n-min 0`): the block-diffusion drafter proposes 4 tokens per step. - **Draft device**: `--spec-draft-device ROCm0` (the drafter runs on the iGPU). - **DFlash2 is a block-diffusion drafter** (not a standalone LM). The drafter has a sliding window of 2048 (~40 MiB of draft KV). ## Chat template The serve script passes `--chat-template-file` pointing at `dankserver/frank-medium/qwen-sharp-v22.4.0.jinja` (the Qwen-Sharp v22.4.0 template, the same one frank-medium uses). Without this flag the binary falls back to the built-in `peg-native` template, which produces a ~73,600-token prompt for a ~250-token request (280x the expected size). With the sharp template wired in, the same request produces a 62–175-token prompt. The `Chat format: peg-native` log line is the format-type label, not the template indicator — the sharp template is active even when that line appears. ## Memory budget The iGPU addresses **114 GiB** (the GTT aperture over the shared LPDDR, `gtt_total` in `/sys/class/drm/card1/device/`; `rocm-smi`'s 2 GiB "dedicated" is not the budget). The 27B dense model (31.46 GiB GGUF) is mmap'd in with `-ngl 999`; budget at 262k ≈ 41–43 GiB (weights + drafter + fp8 KV + buffers), leaving ~71–73 GiB of aperture headroom. The 8 GiB prompt-cache budget (`CACHE_RAM=8192`) is the qualified config. The model is a 27B hybrid (16 full attention + 48 Gated DeltaNet); its DeltaNet state is smaller than a primarily-DeltaNet model, so the fleet's 12.8 GiB DeltaNet `CACHE_RAM` floor does not apply. **KV budget**: 16 full-attention layers × 2 (K+V) × 4 kv_heads × 256 head_dim × 1 byte (fp8) = 32,768 bytes/token. 262,144 tokens × 32,768 ≈ 8 GiB for attention KV. Plus 48 DeltaNet recurrent states (small). ## Payload `scripts/download-frank-large.sh` (resumable, each file verified against its declared byte count + sha256; `--verify-only` re-checks): | file | size | sha256 | |---|---|---| | `Qwen3.8-27B-UD-Q8_K_XL.gguf` | 31,457,991,680 | `af36ecb6…` | | `Qwen3.8-27B-DFlash2-Q4_K_M.gguf` | 1,143,006,816 | `1a25c568…` | ## Client contract - **Port**: 7770 (the same port and alias the prior deployments served). - **Alias**: `frank-large`. - **Context**: 262,144 (the model's native window), 1 slot. - **KV**: fp8 quantized (`--cache-type-k fp8 --cache-type-v fp8`). - **Sampling**: greedy (`--temp 0 --top-p 1.0 --top-k 0 --min-p 0.0`) — the DFlash2 drafter needs greedy verification. - **Reasoning**: enabled (`--reasoning auto --reasoning-format deepseek --reasoning-budget -1`), extracted to `reasoning_content`. - **Vision**: text-only (the model has a vision projector — mmproj-BF16 is available — but this deployment serves text-only, matching the prior frank-large). The fleet's image path is [[frank-small-gemma4-12b]] either way. - **Reasoning effort**: the Qwen3.8 template gates `reasoning_effort` to {xhigh, medium, low} — the harness folds its levels onto that set in `models.yml` (`reasoningEffortMap`). ## Measured performance (To be filled in after deploy.) ## Failure modes worth knowing - **The model has `mtp_num_hidden_layers: 1`** but this deployment uses DFlash2 (not MTP) for spec decode. The MTP layer is unused. - **The model has vision** (`language_model_only: false`, image_token_id 248056, mmproj-BF16 0.93 GB available) but this deployment serves text-only. - **DFlash2 needs greedy verification**: the serve script pins `--temp 0 --top-p 1.0 --top-k 0 --min-p 0.0`. A non-greedy request will not be verified correctly by the drafter. ## Rollback [[frank-large-kairic-edge-qwen38-27b]] — the Kairic Edge payload and runtime are intact on disk.