Frank
4462aa992c
- frank-default-small (Dankserver): Ornith-1.5-35B-A3B-CRACK Q8_0, bundled native draft-mtp (n_max 4), MoE expert FFN tensors pinned to CPU via an explicit --override-tensor \.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU (replaces --n-cpu-moe), Qwen-Sharp chat template. - frank-default-large (Dankclaw): kingjones Qwen3.8-27B-Uncensored-Q6_0 ROCmFP4 AGENT dense GGUF, standalone mtp-Qwen3.8-27B-Q4_0 draft head (--model-draft, draft-mtp n_max 4), Qwen-Sharp chat template. Drop the retired PromptForge .pfs sidecar lanes. - Track scripts/chat-template.jinja (v22.4.0 Sharp) and apply via --chat-template-file on both services. - Docs: new kingjones-strix-mtp and dealign-ornith-crack runbook pages; update index, deployment, and both system specs; models.json names. Smoke-tested both models on the real hardware on scratch ports before commit: model loads, MTP draft attaches, Sharp template applies, chat completion returns.
91 lines
3.0 KiB
Markdown
91 lines
3.0 KiB
Markdown
---
|
|
type: reference
|
|
title: kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
|
description: kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Dense model with a standalone MTP Q4_0 draft head. frank-default-large.
|
|
tags:
|
|
- llama.cpp
|
|
- rocm
|
|
- dankclaw
|
|
- gfx1151
|
|
- kingjones
|
|
- rocmfp4
|
|
- mtp
|
|
timestamp: 2026-08-24
|
|
---
|
|
|
|
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
|
|
|
> **Active 2026-08-24.** `frank-default-large` serves this model. It
|
|
> replaced the jcbtc v3 CIRU ActiveFPX PromptForge payload on the same
|
|
> ROCmFPX runtime and the same CIRU llama.cpp binary. See
|
|
> [[ciru-activefpx-promptforge]] for the retired v3 page.
|
|
|
|
`frank-default-large` (port 7770) serves
|
|
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
|
|
Face. It is a **dense** Qwen3.8-27B, abliterated (uncensored), quantized to
|
|
the ROCmFP4 Q6 AGENT format (ftype 114). The MTP speculative head ships as
|
|
a **separate** Q4_0 draft GGUF, so a bundled-head auto-detect does not apply
|
|
here; the head is loaded with `--model-draft`.
|
|
|
|
## Model Files
|
|
|
|
The card ships six text files plus the vision projector. The service uses
|
|
the Q6 AGENT file (requested for the fleet), the separate MTP draft head
|
|
and the Q8 vision projector.
|
|
|
|
| File | Size |
|
|
|------|------|
|
|
| `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` | 23.15 GiB |
|
|
| `mtp-Qwen3.8-27B-Q4_0.gguf` (draft head) | 1.56 GiB |
|
|
| `mmproj-Qwen3.8-27B-Q8_0.gguf` (vision) | 0.59 GiB |
|
|
|
|
Store the files under `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/`.
|
|
|
|
## Speculative Decoding
|
|
|
|
Dense model, so the finding measured on the dense Qwen3.8 family holds:
|
|
**MTP alone at `--spec-draft-n-max 4` is the knee**. ngram under MTP hurts
|
|
(measured -4 to -5 t/s on the v3; see [[ciru-activefpx-promptforge]]). The
|
|
draft head is a separate file loaded via:
|
|
|
|
```
|
|
--spec-type draft-mtp
|
|
--model-draft mtp-Qwen3.8-27B-Q4_0.gguf
|
|
--spec-draft-n-max 4
|
|
--spec-draft-n-min 0
|
|
--spec-draft-p-min 0.0
|
|
```
|
|
|
|
The draft head loads on `ROCm1` with full offload (`--spec-draft-ngl 999`)
|
|
and f16 KV.
|
|
|
|
## Launch Excerpt
|
|
|
|
The key flags on the CIRU ROCmFPX runtime:
|
|
|
|
```
|
|
llama-server \
|
|
-m Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf \
|
|
--mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
|
|
--chat-template-file chat-template.jinja \
|
|
-ngl 999 -dev ROCm0 -c 262144 -np 1 \
|
|
--spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf \
|
|
--spec-draft-ngl 999 --spec-draft-n-max 4 \
|
|
-fa on --no-mmap --cache-type-k f16 --cache-type-v f16
|
|
```
|
|
|
|
## Known Issues
|
|
|
|
1. **GPU only**: Vulkan, CUDA, and CPU cannot load these files.
|
|
2. `--spec-draft-n-max` defaults to 16; set 4.
|
|
3. `reasoning_effort: "none"` throws in the Qwen API. Use
|
|
`enable_thinking: false` instead.
|
|
4. Small `max_tokens` with thinking on returns empty `content`. The Sharp
|
|
template drives thinking on by default (effort medium).
|
|
|
|
## See Also
|
|
|
|
- [[deployment]] — deploy and update process
|
|
- [[ciru-activefpx-promptforge]] — the retired v3 page
|
|
- [[Dankclaw System Specs]] — the box
|
|
- [[frank-redo-2026-08-17]] — fleet topology history |