Frank
8212d4451a
Mechanical identity rename across the knowledge base, AGENTS.md, README, the
hermes config example, and homepage services.json. No behavior change; served
aliases on the LLM endpoints change with the deploy of commit 69f87c8.
175 lines
6.0 KiB
Markdown
175 lines
6.0 KiB
Markdown
---
|
|
type: reference
|
|
title: DFlash2 on Dankclaw (frank-large)
|
|
description: ARCHIVED - unsloth Qwen3.8-27B-UD-Q8_K_XL served by upstream llama.cpp PR #27342 with a z-lab DFlash2 drafter on the Vulkan backend. Retired 2026-08-21 for speed; kept as a runbook.
|
|
tags:
|
|
- llama.cpp
|
|
- vulkan
|
|
- dankclaw
|
|
- gfx1151
|
|
- dflash
|
|
- dflash2
|
|
- speculative-decoding
|
|
timestamp: 2026-08-21
|
|
---
|
|
|
|
# DFlash2 on Dankclaw (frank-large) — ARCHIVED
|
|
|
|
> **Archived 2026-08-21.** This DFlash2 Vulkan deployment was retired the
|
|
> same day it shipped because it was slower than the CIRU runtime it
|
|
> replaced. `frank-large` is back on CIRU. This page stays as the
|
|
> runbook and provenance for the DFlash2 attempt.
|
|
|
|
`frank-large` (port 52625) briefly served
|
|
`unsloth/Qwen3.8-27B-UD-Q8_K_XL` with a DFlash2 drafter from z-lab. The
|
|
runtime was upstream llama.cpp PR #27342 built with `GGML_VULKAN=ON`.
|
|
|
|
## Why Vulkan
|
|
|
|
DFlash2 selects its draft candidates with a lightweight path selector
|
|
built on the `TOP_K` and `ARGMAX` graph ops. The HIP backend does not
|
|
implement `TOP_K`. That is why the retired CIRU setup routed the MTP
|
|
draft around it with `LLAMA_MTP_CPU_ARGMAX_FASTPATH`.
|
|
|
|
The PR reference fork (Nathanw1014/strix-halo-llamacpp) validated DFlash2
|
|
on this exact gfx1151 through Vulkan (RADV). Its `test-backend-ops` passed
|
|
the new graph ops:
|
|
|
|
| Op | Cases |
|
|
|----|-------|
|
|
| TOP_K | 445 |
|
|
| CONCAT | 193 |
|
|
| PAD | 28 |
|
|
| REPEAT | 18 |
|
|
| FILL | 4 |
|
|
|
|
The whole stack (target + drafter) runs on `Vulkan0`.
|
|
|
|
## What Runs
|
|
|
|
| Artifact | Repo / File | Size |
|
|
|----------|-------------|------|
|
|
| Target | `unsloth/Qwen3.8-27B-GGUF` `Qwen3.8-27B-UD-Q8_K_XL.gguf` | 31.4 GB |
|
|
| Drafter | `z-lab/Qwen3.8-27B-DFlash2-GGUF` `Qwen3.8-27B-DFlash2-Q4_K_M.gguf` | 2.0 GB |
|
|
| Runtime | upstream llama.cpp PR #27342, GGML_VULKAN=ON | - |
|
|
|
|
The target was already on disk. The drafter was added for this deploy.
|
|
Both live in `/home/dank/models/`.
|
|
|
|
## Drafter Hyperparameters
|
|
|
|
Read from the drafter GGUF metadata:
|
|
|
|
| Parameter | Value |
|
|
|-----------|-------|
|
|
| `dflash.block_count` | 5 |
|
|
| `dflash.block_size` | 8 |
|
|
| `dflash.conv_kernel_size` | 2 |
|
|
| `dflash.conv_group_size` | 16 |
|
|
| `dflash.selector_rank` | 256 |
|
|
| `dflash.selector_top_k` | 16 |
|
|
| `dflash.context_length` | 262144 |
|
|
| `dflash.embedding_length` | 5120 |
|
|
| architecture | `dflash` |
|
|
|
|
The target and drafter share the Qwen3.8 architecture
|
|
(`qwen35`, dense 27B, vocab 248320), so the drafter inherits the target
|
|
tokenizer and LM head.
|
|
|
|
## Build Runbook (2026-08-21)
|
|
|
|
Performed on Dankclaw.
|
|
|
|
1. In `/home/dank/llama.cpp`, register the `ggml` remote and fetch the PR
|
|
head:
|
|
```
|
|
git -C /home/dank/llama.cpp remote add ggml https://github.com/ggml-org/llama.cpp.git
|
|
git -C /home/dank/llama.cpp fetch ggml pull/27342/head:pr-27342
|
|
```
|
|
2. Add a detached worktree that does not disturb the running build:
|
|
```
|
|
git -C /home/dank/llama.cpp worktree add --detach /home/dank/llama.cpp-dflash2 pr-27342
|
|
```
|
|
3. Configure with Vulkan only (glslang and `glslc` must be installed):
|
|
```
|
|
cmake -B /home/dank/llama.cpp-dflash2/build-dflash2 -G Ninja \
|
|
-DGGML_VULKAN=ON -DGGML_HIP=OFF -DGGML_CUDA=OFF -DGGML_METAL=OFF \
|
|
-DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release \
|
|
-DLLAMA_BUILD_SERVER=ON -DLLAMA_BUILD_WEBUI=OFF
|
|
```
|
|
4. Build the server:
|
|
```
|
|
cmake --build /home/dank/llama.cpp-dflash2/build-dflash2 --target llama-server -j16
|
|
```
|
|
5. Verify the Vulkan device:
|
|
```
|
|
/home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server --list-devices
|
|
```
|
|
Output: `Vulkan0: AMD Radeon Graphics (RADV STRIX_HALO)`.
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[Fetch PR #27342 head] --> B[Worktree on DFlash2 head]
|
|
B --> C[Configure Vulkan build]
|
|
C --> D[Build llama-server]
|
|
D --> E[Verify Vulkan0 device]
|
|
E --> F[Add DFlash2 drafter]
|
|
F --> G[Swap frank-large]
|
|
G --> H[Health check + completion]
|
|
```
|
|
|
|
## Service
|
|
|
|
- Unit: `frank-large.service`, port 52625
|
|
- Script: `scripts/serve-frank-large.sh`, env `scripts/frank-large.env`
|
|
- Binary: `/home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server`
|
|
|
|
### Launch profile
|
|
|
|
- 262144 context (native max), 1 slot, f16 KV (~16 GiB), 16 context
|
|
checkpoints, 8 GiB prompt cache
|
|
- DFlash2 draft, block size 8, device Vulkan0, drafter fully offloaded
|
|
- `--spec-type draft-dflash --spec-draft-n-max 8`
|
|
(clamped to block size 8 - 1 = 7 at runtime)
|
|
- `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`)
|
|
- `--no-mmap`, flash attention `-fa on`, `--metrics`
|
|
|
|
### Required environment
|
|
|
|
No PromptForge variables are needed. The `TEMP`, `HOST`, and
|
|
`GPU_MAX_HEAP_SIZE` naming rule still applies for portability, even though
|
|
this build uses Vulkan, not HIP.
|
|
|
|
## Validation
|
|
|
|
The smoke test on 2026-08-21 used a scratch port (52626) and confirmed:
|
|
|
|
- Both models load and the server reports healthy.
|
|
- A chat completion returns reasoning and content.
|
|
- The DFlash2 drafter engaged:
|
|
`draft-dflash: #gen drafts = 11, #acc drafts = 10, #mean acc len = 5.27`.
|
|
- The per-position acceptance curve decays along the block
|
|
`(0.909, 0.818, 0.727, 0.545, 0.545, 0.364, 0.364)`, as expected.
|
|
|
|
The published mean acceptance for the Q4_K_M drafter is ~5.39, so 5.27
|
|
is close.
|
|
|
|
## Operational Notes
|
|
|
|
- Health: `curl http://127.0.0.1:52625/health`. Returns 503 while loading.
|
|
- Model load takes about a minute with a warm OS page cache (31 GB target)
|
|
and longer on a cold cache. Do not kill the process during warmup.
|
|
- `deploy.sh dankclaw` waits 20 minutes for health.
|
|
- The Vulkan device reports 113942 MiB. rocm-smi reports the 64 GiB
|
|
unified-memory budget. Trust the runtime for the addressable SVM pool.
|
|
- The MTP head of the target GGUF is ignored (`--spec-type draft-dflash`
|
|
replaces it). The load log warns about unused MTP tensors; this is normal.
|
|
|
|
## See Also
|
|
|
|
- [[ciru-activefpx-promptforge]] — the active CIRU runtime on Dankclaw
|
|
- [[DFlash DSpark Speculative Decoding]] — DFlash/DSpark design and history
|
|
- [[deployment]] — the deployment and update process
|
|
- [[qwen3.8-strix-optimizations]] — llama.cpp Optimizations on this iGPU
|
|
- [[index]] — knowledge base index
|