serverAppConfigs/serverAppConfigs-docs/LLM/dflash2-large-vulkan.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	DFlash2 on Dankclaw (frank-default-large)	ARCHIVED - unsloth Qwen3.8-27B-UD-Q8_K_XL served by upstream llama.cpp PR	llama.cpp vulkan dankclaw gfx1151 dflash dflash2 speculative-decoding	2026-08-21

DFlash2 on Dankclaw (frank-default-large) — ARCHIVED

Archived 2026-08-21. This DFlash2 Vulkan deployment was retired the same day it shipped because it was slower than the CIRU runtime it replaced. frank-default-large is back on CIRU. This page stays as the runbook and provenance for the DFlash2 attempt.

frank-default-large (port 52625) briefly served unsloth/Qwen3.8-27B-UD-Q8_K_XL with a DFlash2 drafter from z-lab. The runtime was upstream llama.cpp PR #27342 built with GGML_VULKAN=ON.

Why Vulkan

DFlash2 selects its draft candidates with a lightweight path selector built on the TOP_K and ARGMAX graph ops. The HIP backend does not implement TOP_K. That is why the retired CIRU setup routed the MTP draft around it with LLAMA_MTP_CPU_ARGMAX_FASTPATH.

The PR reference fork (Nathanw1014/strix-halo-llamacpp) validated DFlash2 on this exact gfx1151 through Vulkan (RADV). Its test-backend-ops passed the new graph ops:

Op	Cases
TOP_K	445
CONCAT	193
PAD	28
REPEAT	18
FILL	4

The whole stack (target + drafter) runs on Vulkan0.

What Runs

Artifact	Repo / File	Size
Target	unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-Q8_K_XL.gguf	31.4 GB
Drafter	z-lab/Qwen3.8-27B-DFlash2-GGUF Qwen3.8-27B-DFlash2-Q4_K_M.gguf	2.0 GB
Runtime	upstream llama.cpp PR #27342, GGML_VULKAN=ON	-

The target was already on disk. The drafter was added for this deploy. Both live in /home/dank/models/.

Drafter Hyperparameters

Read from the drafter GGUF metadata:

Parameter	Value
dflash.block_count	5
dflash.block_size	8
dflash.conv_kernel_size	2
dflash.conv_group_size	16
dflash.selector_rank	256
dflash.selector_top_k	16
dflash.context_length	262144
dflash.embedding_length	5120
architecture	dflash

The target and drafter share the Qwen3.8 architecture (qwen35, dense 27B, vocab 248320), so the drafter inherits the target tokenizer and LM head.

Build Runbook (2026-08-21)

Performed on Dankclaw.

1. In /home/dank/llama.cpp, register the ggml remote and fetch the PR head:

   git -C /home/dank/llama.cpp remote add ggml https://github.com/ggml-org/llama.cpp.git
   git -C /home/dank/llama.cpp fetch ggml pull/27342/head:pr-27342
   

2. Add a detached worktree that does not disturb the running build:

   git -C /home/dank/llama.cpp worktree add --detach /home/dank/llama.cpp-dflash2 pr-27342
   

3. Configure with Vulkan only (glslang and glslc must be installed):

   cmake -B /home/dank/llama.cpp-dflash2/build-dflash2 -G Ninja \
     -DGGML_VULKAN=ON -DGGML_HIP=OFF -DGGML_CUDA=OFF -DGGML_METAL=OFF \
     -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release \
     -DLLAMA_BUILD_SERVER=ON -DLLAMA_BUILD_WEBUI=OFF
   

4. Build the server:

   cmake --build /home/dank/llama.cpp-dflash2/build-dflash2 --target llama-server -j16
   

5. Verify the Vulkan device:

   /home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server --list-devices
   

   Output: Vulkan0: AMD Radeon Graphics (RADV STRIX_HALO).

graph TD
    A[Fetch PR #27342 head] --> B[Worktree on DFlash2 head]
    B --> C[Configure Vulkan build]
    C --> D[Build llama-server]
    D --> E[Verify Vulkan0 device]
    E --> F[Add DFlash2 drafter]
    F --> G[Swap frank-default-large]
    G --> H[Health check + completion]

Service

• Unit: frank-default-large.service, port 52625
• Script: scripts/serve-frank-large.sh, env scripts/frank-default-large.env
• Binary: /home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server

Launch profile

• 262144 context (native max), 1 slot, f16 KV (~16 GiB), 16 context checkpoints, 8 GiB prompt cache
• DFlash2 draft, block size 8, device Vulkan0, drafter fully offloaded
• --spec-type draft-dflash --spec-draft-n-max 8 (clamped to block size 8 - 1 = 7 at runtime)
• --temp 0.7 --top-p 0.95, reasoning on (--reasoning-format deepseek)
• --no-mmap, flash attention -fa on, --metrics

Required environment

No PromptForge variables are needed. The TEMP, HOST, and GPU_MAX_HEAP_SIZE naming rule still applies for portability, even though this build uses Vulkan, not HIP.

Validation

The smoke test on 2026-08-21 used a scratch port (52626) and confirmed:

• Both models load and the server reports healthy.
• A chat completion returns reasoning and content.
• The DFlash2 drafter engaged: draft-dflash: #gen drafts = 11, #acc drafts = 10, #mean acc len = 5.27.
• The per-position acceptance curve decays along the block (0.909, 0.818, 0.727, 0.545, 0.545, 0.364, 0.364), as expected.

The published mean acceptance for the Q4_K_M drafter is ~5.39, so 5.27 is close.

Operational Notes

• Health: curl http://127.0.0.1:52625/health. Returns 503 while loading.
• Model load takes about a minute with a warm OS page cache (31 GB target) and longer on a cold cache. Do not kill the process during warmup.
• deploy.sh dankclaw waits 20 minutes for health.
• The Vulkan device reports 113942 MiB. rocm-smi reports the 64 GiB unified-memory budget. Trust the runtime for the addressable SVM pool.
• The MTP head of the target GGUF is ignored (--spec-type draft-dflash replaces it). The load log warns about unused MTP tensors; this is normal.

See Also

• ciru-activefpx-promptforge — the active CIRU runtime on Dankclaw
• DFlash DSpark Speculative Decoding — DFlash/DSpark design and history
• deployment — the deployment and update process
• qwen3.8-strix-optimizations — llama.cpp Optimizations on this iGPU
• index — knowledge base index