Frank
55f87fcdce
The 2026-10-09 wire aliases become the service names everywhere: systemd units, serve scripts, env files and FRANK_* variables, deploy-plan/deploy-remote service keys, container names, Prometheus job names + alert rules, Grafana dashboard queries, bats seams, runbook filenames, homepage/AGENTS lore, and historical prose. Rollback tags renamed in lockstep. Residue in the tree: zero. Retired frank-medium/small/npu names are history and stay. Known casualty: scripts/patches/*.patch context lines were rewritten; those retired-era vLLM-fork patches no longer apply byte-exact.
101 lines
3.7 KiB
Markdown
101 lines
3.7 KiB
Markdown
---
|
|
type: reference
|
|
title: kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
|
description: kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Dense model with a standalone MTP Q4_0 draft head. frank-dragoon.
|
|
tags:
|
|
- llama.cpp
|
|
- rocm
|
|
- dankclaw
|
|
- gfx1151
|
|
- kingjones
|
|
- rocmfp4
|
|
- mtp
|
|
timestamp: 2026-08-24
|
|
---
|
|
|
|
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
|
|
|
> **RETIRED 2026-08-29.** This payload was `frank-dragoon` from 2026-08-25
|
|
> until the Q8_0 AGENT experiment on 2026-08-29 (it was restored again
|
|
> 2026-09-01 and served until 2026-08-30, when [[agentionai-fast-imatrix]]
|
|
> took over). The Q8_0 model was retired because it crashed before
|
|
> serving. Kept as the historical record of the Q6 AGENT model, standalone
|
|
> MTP draft, Q8 projector, and PromptForge runtime.
|
|
|
|
> This page documents a retired model and its runbook.
|
|
|
|
`frank-dragoon` (port 7770) serves
|
|
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
|
|
Face. It is a **dense** Qwen3.8-27B, abliterated (uncensored), quantized to
|
|
the ROCmFP4 Q6 AGENT format (ftype 114). The MTP speculative head ships as
|
|
a **separate** Q4_0 draft GGUF, so a bundled-head auto-detect does not apply
|
|
here; the head is loaded with `--model-draft`.
|
|
|
|
## Model Files
|
|
|
|
The card ships six text files plus the vision projector. The service uses
|
|
the Q6 AGENT file (requested for the fleet), the separate MTP draft head
|
|
and the Q8 vision projector (`mmproj-Qwen3.8-27B-Q8_0.gguf`).
|
|
|
|
| File | Size |
|
|
|------|------|
|
|
| `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` | 23.15 GiB |
|
|
| `mtp-Qwen3.8-27B-Q4_0.gguf` (draft head) | 1.56 GiB |
|
|
| `mmproj-Qwen3.8-27B-Q8_0.gguf` (vision) | 0.59 GiB |
|
|
|
|
Store the files under `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/`.
|
|
|
|
## Speculative Decoding
|
|
|
|
Dense model, so the finding measured on the dense Qwen3.8 family holds:
|
|
**MTP alone at `--spec-draft-n-max 4` is the knee**. ngram under MTP hurts
|
|
(measured -4 to -5 t/s on the v3; see [[ciru-activefpx-promptforge]]). The
|
|
draft head is a separate file loaded via:
|
|
|
|
```
|
|
--spec-type draft-mtp
|
|
--model-draft mtp-Qwen3.8-27B-Q4_0.gguf
|
|
--spec-draft-n-max 4
|
|
--spec-draft-n-min 0
|
|
--spec-draft-p-min 0.0
|
|
```
|
|
|
|
The draft head loads on `ROCm1` with full offload (`--spec-draft-ngl 999`)
|
|
and f16 KV.
|
|
|
|
## Launch Excerpt
|
|
|
|
The key flags on the CIRU ROCmFPX runtime:
|
|
|
|
```
|
|
llama-server \
|
|
-m Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf \
|
|
--mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
|
|
--chat-template-file chat-template.jinja \
|
|
--chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
|
|
-ngl 999 -dev ROCm0 -c 196608 -np 1 \
|
|
-b 8192 -ub 8192 -ctxcp 32 \
|
|
--spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf \
|
|
--spec-draft-ngl 999 --spec-draft-n-max 4 \
|
|
-fa on --no-mmap --cache-type-k f16 --cache-type-v f16
|
|
```
|
|
|
|
## Known Issues
|
|
|
|
1. **GPU only**: Vulkan, CUDA, and CPU cannot load these files.
|
|
2. `--spec-draft-n-max` defaults to 16; set 4.
|
|
3. `reasoning_effort: "none"` throws in the Qwen API. Use
|
|
`enable_thinking: false` instead.
|
|
4. Small `max_tokens` with thinking on returns empty `content`. The Sharp
|
|
template drives thinking on by default (effort medium).
|
|
5. `preserve_thinking:true` (in `chat_template_kwargs`) keeps `<think>`
|
|
blocks in recorded assistant messages. `preserve_thinking:false` strips
|
|
them. See [[deployment]] for the chat_format/reasoning_format pairing
|
|
(`peg-native` + `deepseek`) the Pi agent expects from this endpoint.
|
|
|
|
## See Also
|
|
|
|
- [[deployment]] — deploy and update process
|
|
- [[ciru-activefpx-promptforge]] — the retired v3 page
|
|
- [[Dankclaw System Specs]] — the box
|
|
- [[frank-redo-2026-08-17]] — fleet topology history |