Frank
7ca9dc248e
Replace the retired kingjones ROCmFP4 STRIX MTP payload on Dankclaw with cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF (eemin namespace, public), served by the charlie12345/ROCmFPX llama.cpp build (ROCmI4 W4A4, c49ebdbd) on the Strix Halo iGPU. Q4_0_ROCMI4 with an embedded NextN/MTP head (no separate draft file); strict-Qwen MTP n16/p0.60/backend sampling on ROCm0. Binary /home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server; model dir /home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP. Updated serve script + env, ubatch bench, large-config bats test, docs, AGENTS.md.
100 lines
3.6 KiB
Markdown
100 lines
3.6 KiB
Markdown
---
|
|
type: reference
|
|
title: kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
|
description: kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Dense model with a standalone MTP Q4_0 draft head. frank-default-large.
|
|
tags:
|
|
- llama.cpp
|
|
- rocm
|
|
- dankclaw
|
|
- gfx1151
|
|
- kingjones
|
|
- rocmfp4
|
|
- mtp
|
|
timestamp: 2026-08-24
|
|
---
|
|
|
|
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
|
|
|
> **Retired 2026-08-25.** The kingjones ROCmFP4 STRIX MTP payload has been
|
|
> replaced by the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on the same
|
|
> port and iGPU, under a new runtime (charlie12345/ROCmFPX ROCmI4 W4A4). See
|
|
> [[eemin-rocmi4-mtp]]. The old kingjones GGUF + standalone MTP Q4_0 draft are
|
|
> retired on this slot. The CIRU ROCmFPX runtime/binary is also retired here.
|
|
|
|
> _This page documents the retired model and its runbook for reference._
|
|
|
|
`frank-default-large` (port 7770) serves
|
|
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
|
|
Face. It is a **dense** Qwen3.8-27B, abliterated (uncensored), quantized to
|
|
the ROCmFP4 Q6 AGENT format (ftype 114). The MTP speculative head ships as
|
|
a **separate** Q4_0 draft GGUF, so a bundled-head auto-detect does not apply
|
|
here; the head is loaded with `--model-draft`.
|
|
|
|
## Model Files
|
|
|
|
The card ships six text files plus the vision projector. The service uses
|
|
the Q6 AGENT file (requested for the fleet), the separate MTP draft head
|
|
and the Q8 vision projector.
|
|
|
|
| File | Size |
|
|
|------|------|
|
|
| `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` | 23.15 GiB |
|
|
| `mtp-Qwen3.8-27B-Q4_0.gguf` (draft head) | 1.56 GiB |
|
|
| `mmproj-Qwen3.8-27B-Q8_0.gguf` (vision) | 0.59 GiB |
|
|
|
|
Store the files under `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/`.
|
|
|
|
## Speculative Decoding
|
|
|
|
Dense model, so the finding measured on the dense Qwen3.8 family holds:
|
|
**MTP alone at `--spec-draft-n-max 4` is the knee**. ngram under MTP hurts
|
|
(measured -4 to -5 t/s on the v3; see [[ciru-activefpx-promptforge]]). The
|
|
draft head is a separate file loaded via:
|
|
|
|
```
|
|
--spec-type draft-mtp
|
|
--model-draft mtp-Qwen3.8-27B-Q4_0.gguf
|
|
--spec-draft-n-max 4
|
|
--spec-draft-n-min 0
|
|
--spec-draft-p-min 0.0
|
|
```
|
|
|
|
The draft head loads on `ROCm1` with full offload (`--spec-draft-ngl 999`)
|
|
and f16 KV.
|
|
|
|
## Launch Excerpt
|
|
|
|
The key flags on the CIRU ROCmFPX runtime:
|
|
|
|
```
|
|
llama-server \
|
|
-m Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf \
|
|
--mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
|
|
--chat-template-file chat-template.jinja \
|
|
--chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
|
|
-ngl 999 -dev ROCm0 -c 196608 -np 1 \
|
|
-b 8192 -ub 8192 -ctxcp 32 \
|
|
--spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf \
|
|
--spec-draft-ngl 999 --spec-draft-n-max 4 \
|
|
-fa on --no-mmap --cache-type-k f16 --cache-type-v f16
|
|
```
|
|
|
|
## Known Issues
|
|
|
|
1. **GPU only**: Vulkan, CUDA, and CPU cannot load these files.
|
|
2. `--spec-draft-n-max` defaults to 16; set 4.
|
|
3. `reasoning_effort: "none"` throws in the Qwen API. Use
|
|
`enable_thinking: false` instead.
|
|
4. Small `max_tokens` with thinking on returns empty `content`. The Sharp
|
|
template drives thinking on by default (effort medium).
|
|
5. `preserve_thinking:true` (in `chat_template_kwargs`) keeps `<think>`
|
|
blocks in recorded assistant messages. `preserve_thinking:false` strips
|
|
them. See [[deployment]] for the chat_format/reasoning_format pairing
|
|
(`peg-native` + `deepseek`) the Pi agent expects from this endpoint.
|
|
|
|
## See Also
|
|
|
|
- [[deployment]] — deploy and update process
|
|
- [[ciru-activefpx-promptforge]] — the retired v3 page
|
|
- [[Dankclaw System Specs]] — the box
|
|
- [[frank-redo-2026-08-17]] — fleet topology history |