dank/serverAppConfigs
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Swap frank-default-large to DFlash2 on Vulkan (PR #27342)

Browse Source 
Replace the retired CIRU ROCmFPX runtime with upstream llama.cpp PR #27342
DFlash2 on the Vulkan backend:

- Target: unsloth Qwen3.8-27B-UD-Q8_K_XL.gguf (31.4 GB, already on disk)
- Drafter: z-lab Qwen3.8-27B-DFlash2-Q4_K_M.gguf (block size 8, top-k 16)
- Runtime: /home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server
  (GGML_VULKAN=ON; HIP lacks TOP_K so the stack runs on Vulkan0/RADV)
- spec: --spec-type draft-dflash --spec-draft-n-max 8 (clamps to 7)

Validated 2026-08-21 on scratch port 52626: mean acceptance 5.27,
health + reasoning completion OK.

Archive the retired CIRU .env/.serve script in .AGENTTRASH.
Update deployment.md, index.md, AGENTS.md; add docs/dflash2-large-vulkan.md.
This commit is contained in:
Frank
2026-08-21 09:40:08 -07:00
parent 701e742f9f
commit fd0d9dd3cb
9 changed files with 470 additions and 136 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/deployment.md
+11 -11
View File
@@ -9,7 +9,7 @@ Two servers run LLM inference services. Each server pulls its scripts from this
| Target | Host | Runtime | Service | Port |
|--------|------|---------|---------|------|
| Dankserver | dank@dankserver | upstream llama.cpp (Qwen3.6-35B-A3B Q6_K_P + ngram-mod, MoE on CPU) | frank-default-small | 7770 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP + PromptForge DualView, 262K ctx / 1 slot) | frank-default-large | 52625 |
| Dankclaw | dank@dankclaw | upstream llama.cpp PR #27342 DFlash2 (unsloth Qwen3.8-27B-UD-Q8_K_XL + z-lab DFlash2 Q4_K_M drafter, Vulkan, 262K ctx / 1 slot) | frank-default-large | 52625 |
## Workflow
@@ -115,19 +115,19 @@ launches through `scripts/serve-frank-small.sh`, which reads tunables from
### Dankclaw — frank-default-large
The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` (v2 GGUF + FFN/GDN
companions) under the alias `frank-default-large` on port 52625. It runs
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context
(1 slot; concurrent requests queue, the server never preempts a running
generation), batch 4096 / ubatch 4096, 16
context checkpoints, 8 GiB
The active service on Dankclaw is upstream llama.cpp PR #27342 (DFlash2
support) built with GGML_VULKAN=ON, serving
`/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL.gguf` (unsloth Dynamic v3.0,
31.4 GB) under the alias `frank-default-large` on port 52625. It runs
`/home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server` with a
z-lab DFlash2 drafter (`/home/dank/models/Qwen3.8-27B-DFlash2-Q4_K_M.gguf`,
block size 8) via `--spec-type draft-dflash`, on the Vulkan (RADV) backend
(`Vulkan0`, gfx1151, 113942 MiB). f16 KV for both K and V, full 262144
context (1 slot), batch 4096 / ubatch 4096, 16 context checkpoints, 8 GiB
prompt cache, and reasoning in deepseek format. The unit file is
`scripts/frank-default-large.service`. It launches through
`scripts/serve-frank-large.sh`, which reads tunables from
`scripts/frank-default-large.env`. See [[ciru-activefpx-promptforge]].
`scripts/frank-default-large.env`. See [[dflash2-large-vulkan]].
## Validation