serverAppConfigs/serverAppConfigs-docs/LLM/frank-large-unsloth-qwen38-27b-q8xl.md

---
type: reference
title: stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)
description: The active frank-large runbook — the stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, UD-Q8_K_XL) with the DFlash2 drafter (z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M) on the Dankclaw Strix Halo iGPU: the prebuilt runtime, the full-window/one-stream shape, the DFlash2 spec method, fp8 KV, the Qwen-Sharp template, the payload, the client contract, and the failure modes worth knowing.
tags:
  - llm
  - llama.cpp
  - unsloth
  - dankclaw
  - frank-large
  - strix-halo
  - gfx1151
  - q8
  - dflash2
  - rocm
  - fp8-kv
timestamp: 2026-09-21
---

# stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)

> **ACTIVE since 2026-09-21.** It replaced the Kairic Edge runtime
> (ciru-ai/ROCmFPX, jcbtc/Qwen3.8-27B-IU4-Kairic-Edge — see
> [[frank-large-kairic-edge-qwen38-27b]], the current rollback target),
> which had replaced the unsloth llama.cpp prebuilt serving
> unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (125B MoE — see
> [[frank-large-unsloth-qwen38-flash-next]]). Deployed as a **root
> systemd unit** (`scripts/frank-large.service`, `User=dank`), port 7770,
> on the same gfx1151 iGPU. The RTX 3090 it does not touch belongs to
> [[frank-small-gemma4-12b]].

The model is `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL` — a **27B dense**
Qwen3.8 at **UD-Q8_K_XL** (a high-precision 8-bit quant). It is a qwen3_5
hybrid: 64 layers total, 16 full attention + 48 Gated DeltaNet linear
attention. The payload is two files: the 31.46 GiB main GGUF and the 1.14
GiB DFlash2 Q4_K_M drafter.

The runtime is the **stock llama.cpp** prebuilt (unsloth b11030, ROCm
gfx1151) at `/home/dank/GitPackages/unsloth-llama-b11030/llama-server`.
It is a mainline llama.cpp build that supports the qwen3_5 architecture
and the DFlash2 drafter (llama_model_qwen35, llama_model_dflash,
common_speculative_impl_draft_dflash).

## Why this swap

- The operator asked for the 27B model at **one conversation, full
  context (262,144)** with DFlash2 spec decode (2026-09-21).
- The stock llama.cpp prebuilt is the only stock-compatible runtime with
  qwen3_5 + DFlash2 support — no source build, no CK `config.h`, no
  PromptForge sidecars, no compatibility mode.
- The spec method is DFlash2 (a block-diffusion drafter) at depth 4. The
  drafter has a sliding window of 2048 (small draft KV, ~40 MiB).

## Shape

One root unit, one process, one published port:

| half | port | reachable |
|---|---|---|
| llama-server (OpenAI front-end + engine, single process) | 7770 | published as host **7770** |

## Runtime

- **Prebuilt**: `/home/dank/GitPackages/unsloth-llama-b11030/llama-server`
  (mainline, ROCm gfx1151 prebuilt, version 0.4.1-dev, build 11030,
  commit 6ba30d05b). The build bundles its own ROCm runtime — no host SDK
  gate, no venv.
- **GPU target**: `gfx1151` (Strix Halo).
- **Spec-draft flags**: `--spec-draft-type-k`, `--spec-draft-type-v`,
  `--spec-draft-threads`, `--spec-draft-cpu-mask`, `--spec-draft-prio`,
  `--spec-draft-poll` all present (confirmed via `--help`).

## Spec method

- **DFlash2 depth 4** (`--spec-type draft-dflash --model-draft <drafter>
  --spec-draft-n-max 4 --spec-draft-n-min 0`): the block-diffusion
  drafter proposes 4 tokens per step.
- **Draft device**: `--spec-draft-device ROCm0` (the drafter runs on the
  iGPU).
- **DFlash2 is a block-diffusion drafter** (not a standalone LM). The
  drafter has a sliding window of 2048 (~40 MiB of draft KV).

## Chat template

The serve script passes `--chat-template-file` pointing at
`dankserver/frank-medium/qwen-sharp-v22.4.0.jinja` (the Qwen-Sharp
v22.4.0 template, the same one frank-medium uses). Without this flag the
binary falls back to the built-in `peg-native` template, which produces a
~73,600-token prompt for a ~250-token request (280x the expected size).
With the sharp template wired in, the same request produces a 62–175-token
prompt. The `Chat format: peg-native` log line is the format-type label,
not the template indicator — the sharp template is active even when that
line appears.

## Memory budget

The iGPU addresses **114 GiB** (the GTT aperture over the shared LPDDR,
`gtt_total` in `/sys/class/drm/card1/device/`; `rocm-smi`'s 2 GiB
"dedicated" is not the budget). The 27B dense model (31.46 GiB GGUF) is
mmap'd in with `-ngl 999`; budget at 262k ≈ 41–43 GiB (weights + drafter
+ fp8 KV + buffers), leaving ~71–73 GiB of aperture headroom. The 8 GiB
prompt-cache budget (`CACHE_RAM=8192`) is the qualified config. The model
is a 27B hybrid (16 full attention + 48 Gated DeltaNet); its DeltaNet
state is smaller than a primarily-DeltaNet model, so the fleet's 12.8
GiB DeltaNet `CACHE_RAM` floor does not apply.

**KV budget**: 16 full-attention layers × 2 (K+V) × 4 kv_heads × 256
head_dim × 1 byte (fp8) = 32,768 bytes/token. 262,144 tokens × 32,768 ≈
8 GiB for attention KV. Plus 48 DeltaNet recurrent states (small).

## Payload

`scripts/download-frank-large.sh` (resumable, each file verified against
its declared byte count + sha256; `--verify-only` re-checks):

| file | size | sha256 |
|---|---|---|
| `Qwen3.8-27B-UD-Q8_K_XL.gguf` | 31,457,991,680 | `af36ecb6…` |
| `Qwen3.8-27B-DFlash2-Q4_K_M.gguf` | 1,143,006,816 | `1a25c568…` |

## Client contract

- **Port**: 7770 (the same port and alias the prior deployments served).
- **Alias**: `frank-large`.
- **Context**: 262,144 (the model's native window), 1 slot.
- **KV**: fp8 quantized (`--cache-type-k fp8 --cache-type-v fp8`).
- **Sampling**: greedy (`--temp 0 --top-p 1.0 --top-k 0 --min-p 0.0`) —
  the DFlash2 drafter needs greedy verification.
- **Reasoning**: enabled (`--reasoning auto --reasoning-format deepseek
  --reasoning-budget -1`), extracted to `reasoning_content`.
- **Vision**: text-only (the model has a vision projector — mmproj-BF16
  is available — but this deployment serves text-only, matching the prior
  frank-large). The fleet's image path is [[frank-small-gemma4-12b]]
  either way.
- **Reasoning effort**: the Qwen3.8 template gates `reasoning_effort` to
  {xhigh, medium, low} — the harness folds its levels onto that set in
  `models.yml` (`reasoningEffortMap`).

## Measured performance

(To be filled in after deploy.)

## Failure modes worth knowing

- **The model has `mtp_num_hidden_layers: 1`** but this deployment uses
  DFlash2 (not MTP) for spec decode. The MTP layer is unused.
- **The model has vision** (`language_model_only: false`, image_token_id
  248056, mmproj-BF16 0.93 GB available) but this deployment serves
  text-only.
- **DFlash2 needs greedy verification**: the serve script pins
  `--temp 0 --top-p 1.0 --top-k 0 --min-p 0.0`. A non-greedy request
  will not be verified correctly by the drafter.

## Rollback

[[frank-large-kairic-edge-qwen38-27b]] — the Kairic Edge payload and
runtime are intact on disk.