---
type: reference
title: caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw
description: caf-eemin/Qwen3.8-27B-ROCmI4-MTP-GGUF (Q4_0_ROCMI4, embedded NextN/MTP head) served by the charlie12345/ROCmFPX llama.cpp runtime (ROCmI4 W4A4) on the Strix Halo iGPU. frank-dragoon.
tags:
  - llama.cpp
  - rocm
  - dankclaw
  - gfx1151
  - rocmi4
  - mtp
  - iu4
timestamp: 2026-08-25
---

# caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw

> **RETIRED 2026-08-25.** `frank-dragoon` served this model from 2026-08-18
> until 2026-08-25. It replaced the kingjones777 ROCmFP4 STRIX MTP payload
> on the same port and the same Strix Halo iGPU, running the new
> charlie12345/ROCmFPX ROCmI4 W4A4 build (see [[kingjones-strix-mtp]] for
> the page it replaced), and was itself replaced by
> [[kingjones-strix-rocmfp4]].

`frank-dragoon` (port 7770) serves
`cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (published under the public
`eemin` namespace) from the **charlie12345/ROCmFPX** llama.cpp fork built with
the opt-in `ROCmI4 W4A4` path. The model is `Q4_0_ROCMI4`: signed four-bit
weights packed two per byte, quantized from the BF16 Qwen3.8-27B and carrying
its **embedded NextN/MTP head** in the main GGUF — so there is no separate
draft file. The native gfx1151 `v_wmma_i32_16x16x16_iu4` matrix instruction
batch-verifies the MTP proposals for higher decode throughput.

## What This Is

**ROCmI4** is the GGUF weight format (signed 4-bit codes). **IU4** is the AMD
WMMA instruction spelling that consumes them. **W4A4** means four-bit weights
and four-bit activations during the accelerated matrix multiply. Ordinary
one-token decode stays memory-bandwidth-bound (MMVQ); batched prompt processing
and batched MTP target verification use the IU4 MMQ path. W4A4 is a lossy
activation-quant speed path — treat it as a speed/quality tradeoff.

The model is a conversion/quantization of `Qwen/Qwen3.8-27B` (Unsloth lineage,
Apache-2.0), not a retrained model. It reports architecture `qwen35`
(M-RoPE), ~27.32B params, native 262144 context, 866 tensors, ftype 108.

## Model Files

| File | Size | SHA-256 |
|------|------|---------|
| `Qwen3.8-27B-Q4_0_ROCMI4.gguf` | 13.50 GiB (14,534,384,384 B) | `ad51ecbee55930ababc629c377943a7bbe50f8aafcf32340c7cdae6fa6b7bb3c` |
| `mmproj-BF16.gguf` (vision, optional) | 0.87 GiB (931,146,432 B) | `83ee4f4f205fa514161778c41df1ea14144faa0f713510893b63c2395f5c2d53` |

Store under `/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP/`. The vision
projector is optional for text-only use.

## Runtime Build

The stock llama.cpp build does not understand `Q4_0_ROCMI4` or the qualified
IU4/MTP path. Use charlie12345/ROCmFPX at the pinned commit, built HIP-only
(so the combined HIP/Vulkan scheduler cannot route the ROCmI4 tensors to
Vulkan, which has no shader for this type) with the W4A4 option on.

| Artifact | Path |
|----------|------|
| Fork + pinned commit | `/home/dank/rocmi4-qwen38/ROCmFPX` @ `c49ebdbd5c9f01ec242369f9e7f7967855f80cba` |
| Build | `cmake -S . -B build-rocmi4-w4a4 -G Ninja -DGGML_HIP=ON -DGGML_VULKAN=OFF -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DGGML_HIP_ROCMI4_W4A4=ON -DLLAMA_BUILD_SERVER=ON -DLLAMA_BUILD_WEBUI=OFF` |
| Binary | `/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server` (version 257, `c49ebdb`) |
| ROCm | system `/opt/rocm-7.2.4`, amdclang 22 |

Build a separate directory so the exact-int8 path (the default branch) stays
available. Rollback: disable `-DGGML_HIP_ROCMI4_W4A4=ON` and rebuild — no model
conversion or data change needed. The build prints
`ROCmI4 W4A4: enabled for device 0` when the accelerated path is active.

## Model Download

Downloaded from the public
`eemin/Qwen3.8-27B-ROCmI4-MTP-GGUF` repo (the card's download section still
names the still-private `cafonz/…`; use `eemin`). Verify against the repo's
`checksums.sha256` after download:

```
cd /home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP
sha256sum -c checksums.sha256
```

## Launch Profile

The qualified MTP profile (from the model card and PR #94, verified on the
live service): strict Qwen MTP, backend sampling, `n_max 4`, `p_min 0.60`.

```mermaid
graph TD
    A[llama-server charlie12345 ROCmFPX c49ebdb<br/>ROCmI4 W4A4 enabled] --> B[load Q4_0_ROCMI4 GGUF + mmproj<br/>-ngl 999 -dev ROCm0]
    B --> C[draft-mtp on ROCm0<br/>--spec-draft-ngl all]
    C --> D[strict Qwen MTP<br/>n_max 4 / p_min 0.60<br/>backend sampling]
    D --> E[Sharp chat template<br/>reasoning deepseek]
```

Key flags (see `scripts/serve-frank-dragoon.sh`):

```
--spec-type draft-mtp --spec-mtp-strict-qwen
--spec-draft-device ROCm0 --spec-draft-ngl all
--spec-draft-type-k f16 --spec-draft-type-v f16
--spec-draft-n-max 4 --spec-draft-n-min 0
--spec-draft-p-min 0.60 --spec-draft-backend-sampling
```

Because the MTP head is **embedded**, no `--model-draft` is passed and the
draft uses all layers on ROCm0. `--spec-mtp-strict-qwen` preserves the target
path's greedy decisions; it does not remove the W4A4 activation approximation.

### Serving settings

- 65536 context (192K below the native 262144 maximum), 1 slot, f16 KV,
  context checkpoints, prompt cache. The lower limit leaves headroom for the
  model plus KV cache in the host's unified-memory budget. Concurrent requests
  queue; the server never preempts a running generation.
- `-t 16 -tb 32`, batch 512 / ubatch 256 (the qualified W4A4 profile).
- `--temp 0.7 --top-p 0.95`, reasoning on (`deepseek` puts `<think>` into
  `message.reasoning_content`).
- Qwen-Sharp chat template (`--chat-template-file`), `enable_thinking` +
  `preserve_thinking` on, preserving the peg-native + deepseek shape the Pi
  agent expects.
- Strix Halo unified memory + native gfx1151 detection: `HSA_OVERRIDE_GFX_VERSION=11.5.1`,
  `GGML_HIP_ENABLE_UNIFIED_MEMORY=1`.

### Service

- Unit: `frank-dragoon.service`, port 7770
- Script: `scripts/serve-frank-dragoon.sh`, env `scripts/frank-dragoon.env`

## Tuning MTP

MTP gains are content-dependent: predictable code, structured data, and lists
accept longer drafts than creative prose. The active conservative profile is
`n4 / p0.60`; the previously qualified starting point was `n16 / p0.60`.

## ubatch Benchmark

Decode throughput depends on the ubatch (per-eval token batch). Run the sweep
on 8k contexts, 512 → 4096 in steps of 512, with a fixed batch:

```
sudo systemctl stop frank-dragoon
./scripts/bench-ubatch.sh
sudo systemctl start frank-dragoon
```

The script stops `frank-dragoon` for a clean GPU, boots one server
instance per ubatch on a scratch port, reads authoritative `print_timing` tok/s,
and restarts the service. See [[frank-dragoon-benchmark]].

## Known Issues

1. **gfx1151 only** for the W4A4 path. Other GPUs fall back to exact int8 MMQ.
2. **Memory pressure:** the model is 13.5 GiB. Under `--no-mmap` with full
   offload it needs the whole tensor set at once; leave room in the 64 GiB
   unified pool (rocm-smi figure). Do not run a bench with the live service up
   or it OOMs at load.
3. **`Q4_0_ROCMI4` + MTP flags need the ROCmFPX fork**; stock llama.cpp may
   reject the file or drop the optimized path.
4. **W4A4 is lossy** (+~5.4% perplexity vs exact int8). Opt-in speed path.

## Measured Results (qualification)

From the model card, gfx1151, pinned build, W4A4 + strict MTP-16:

| Result | Value |
|--------|-------|
| HumanEval pilot, W4A4 IU4 (mean) | 49.40 tok/s |
| Full HumanEval, W4A4 (mean / median) | 44.39 / 45.23 tok/s |
| HumanEval / HumanEval+ pass@1 | 94.5% / 91.5% |
| Non-speculative tg128 | ~13.8 tok/s |

Local smoke test 2026-08-25 on the iGPU: model loaded, strict-MTP active,
draft acceptance 0.74 (mean 7.67), generation ~25 tok/s at a small output.
The ubatch sweep above gives the production ubatch choice.

## See Also

- [[deployment]] — deploy and update process
- [[kingjones-strix-mtp]] — the retired ROCmFP4 predecessor on this slot
- [[ciru-activefpx-promptforge]] — the retired v3 CIRU PromptForge page
- [[Dankclaw System Specs]] — the box
- [[frank-dragoon-benchmark]] — backend/spec benchmark history