---
type: reference
title: agentionai Qwen3.8-Flash-Next ROCmFP4-FAST imatrix on Dankclaw
description: agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF served by the charlie12345/ROCmFPX llama.cpp runtime (build-qwen38). 87.06 GiB Q4_0_ROCMFP4_FAST imatrix (v2 joined PLE layout), own MTP draft head (ROCmFP4-FAST, 2.28 GiB). frank-large.
tags:
  - llama.cpp
  - rocm
  - dankclaw
  - gfx1151
  - agentionai
  - rocmfp4
  - strix-halo
  - qwen4exp
  - mtp
timestamp: 2026-08-29
---

# agentionai Qwen3.8-Flash-Next ROCmFP4-FAST imatrix on Dankclaw

> **RETIRED 2026-08-30** — replaced by the kingjones STRIX model on the
> card's pinned kingjones30/ROCmFPX runtime; see
> [[kingjones-strix-rocmfp4]] for the active runbook. This page stays as
> the deployment record: the v2-joined-PLE layout requirement, the MTP
> head incompatibility (verified 3x), and the QSA F32-mask patch
> (`scripts/patches/qwen4exp-qsa-f32-mask.patch`) that made
> `--flash-attn on` work on the charlie build-qwen38 tree — all still
> true for that binary + model pair, which remains the rollback target.

`frank-large` (port 7770) served
`agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF` (main,
`Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf`, the `v2/` joined layout)
with the author's own MTP draft head
(`Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf`, never loaded) on the
charlie12345/ROCmFPX `build-qwen38` runtime, 2026-08-29 → 2026-08-30.

## Why this model

- 87.06 GiB at 4.23 bpw — everything on the GPU of the 128 GB unified-memory
  box, including the 51.2 B-parameter n-gram (PLE) table. REPO `v2/` layout
  is used (joined single `per_layer_token_embd.weight` tensor) because the
  charlie12345/ROCmFPX loader requires it; the root `-ple16` file's per-head
  split only loads in the LaurentZuijdwijk Vulkan fork (see Known issues #2).
- imatrix-calibrated on 1540 chunks; +2.48% PPL vs reference on wikitext-2
  (better than AesSedai IQ4_XS / IQ3_S at smaller size).
- 18-25% faster prompt processing than a mainline quant of similar size
  (356 vs 285 t/s at 2048 tok; 301 vs 256 at 8k depth).
- Ships the model's own f16 vision tower (`mmproj/`).
- The MTP head is quantized to match the target (ROCmFP4-FAST) — a Q8_0
  draft measured worse on this head family — but it does not load on
  build-qwen38 (Known issues #3).

## Runtime

charlie12345/ROCmFPX `llama-server` built at
`/home/dank/GitPackages/ROCmFPX/build-qwen38` (HIP, ROCm 7.2.4, gfx1151,
PR #27742 qwen4exp merged). Upstream llama.cpp cannot load the ROCmFP4
quant types (`TYPE_100`/`TYPE_101`/FAST variants) — verified:

```
strings build-qwen38/bin/llama-server | grep Q4_0_ROCMFP4_FAST
```

CIRU binaries (ciru-ai v1.1) do **not** carry the FAST types; do not point
this model at the CIRU runtime. The build previously served the kingjones
STRIX / STRIX_LEAN models; the agentionai-FAST pair is the first use of the
author's own matched MTP head on this binary.

## Files / download

| Item | Path on Dankclaw | Size |
|---|---|---|
| Main (v2 joined) | `/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/v2/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf` | 93,484,242,816 B (87.06 GiB) |
| Main (root ple16, unused) | `/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2-ple16.gguf` | 93,484,237,760 B (87.06 GiB) |
| Vision | `…/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf` | 904,004,128 B |
| Draft head | `/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf` | 2,444,519,296 B |

Draft head sha256 (verified against HF LFS):
`5046d69571bb35c699e19c7a00b36633c2fdfdd06d70482551744ec523a6b590`

Downloaded with a resumable plain `curl -L -C -` (Hugging Face `resolve/main`
URLs; no auth token needed for public repos). The mmproj lives in an `mmproj/`
subdirectory of the repo.

## Serving profile

Card sampling table (instruct, non-thinking) mixed with the validated
build-qwen38 profile:

```
llama-server \
  --model Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf \
  --mmproj mmproj-Qwen3.8-Flash-Next-f16.gguf \
  --host 0.0.0.0 --port 7770 --jinja \
  --device ROCm0 --gpu-layers 999 \
  --split-mode none --fit off --flash-attn on \
  --ctx-size 32768 --parallel 1 \
  --batch-size 4096 --ubatch-size 4096 \
  --cache-type-k f16 --cache-type-v f16 \
  --threads 16 --threads-batch 16 \
  --ctx-checkpoints 16 --cache-ram 8192 \
  --cache-prompt --cache-idle-slots --metrics \
  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0 --presence-penalty 1.5 \
  --reasoning on --reasoning-format deepseek --reasoning-budget -1 \
  --chat-template-file chat-template.jinja \
  --chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}' \
  --spec-type ngram-mod
```

Key choices and deviations from the CIRU deployment:

- **`--fit off`** — auto-fit segfaults on qwen4exp hyper-connection config
  (same as every Flash-Next deployment).
- **`--checkpoint-min-step` removed** — build-qwen38 does not implement it
  (verified via `--help`); only `--ctx-checkpoints` is passed.
- **No PLE sidecar args** — the n-gram table is in-GGUF (joined
  `per_layer_token_embd.weight` from the `v2/` layout), not a `ple/`
  CIRUPLE1 sidecar. `--ple-sidecar`/`--ple-cache-mib` and the
  `GGML_QWEN4EXP_PLE_*` / `GGML_CUDA_Q41_MOE_FORCE_J` env knobs are CIRU-
  runtime-only and are gone.
- **`--mmproj` active** — vision tower included (f16, lossless).
- **`--flash-attn on` (restored 2026-08-29).** fa on previously aborted graph
  build: the QSA indexer path (layers with attention compression) called
  `ggml_fill` on the F16-cast kq mask (`GGML_ASSERT(a->type == GGML_TYPE_F32)`,
  ggml.c:5371, backtrace through `build_attn_qsa`). Fixed in the fork source
  by the QSA F32-mask patch — vendored at
  `scripts/patches/qwen4exp-qsa-f32-mask.patch` (sha256
  `f2f4a486e21cfad80bfa42683281d18e1cba22fa43c5e0e2aa0cfdbe141e925f`);
  the binary at `build-qwen38/bin/llama-server` (rebuilt 2026-08-29) already
  carries it. A clean rebuild must re-apply
  `git apply scripts/patches/qwen4exp-qsa-f32-mask.patch` inside
  `/home/dank/GitPackages/ROCmFPX` before `make`. Verified live at
  CTX_SIZE=32768: model loads, health 200, real completion.
- Context capped at **32768 × 1 slot — the author's own quick-start**. This
  model is fully GPU-resident (87.06 GiB): the PLE n-gram table sits inside
  the GGUF as one tensor, unlike STRIX_LEAN which streamed it from RAM.
  HIP heap is 119.3 GiB; the context buffer is 50.8 GiB at 131072 and
  103 GiB at 262144, so either window OOMs (`cudaMalloc … failed: out of
  memory`, verified live on first graph alloc). At 32768 the buffer is
  ~13 GiB (~100 GiB total) and loads clean.
- MTP is disabled (ngram-mod only) — see the MTP section below.

## MTP draft head — INCOMPATIBLE on build-qwen38

The author's own matched head (`agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-
FAST-GGUF`, 2.28 GiB, `qwen4exp` arch) does **not** load on this binary:

```
error loading model: missing tensor 'blk.0.hc_attn_norm.weight'
```

The build-qwen38 qwen4exp loader requires a full per-layer
`blk.<il>.hc_attn_norm.weight` set in the draft (the graph reads all 48
layers); the head only carries `blk.48.hc_attn_norm.weight` (count 1 — all
three Flash-Next-era heads examined on this fork have that same partial
layout). Verified three times, not a cross-vendor artifact:

1. EasiiX Q8_0 head vs kingjones STRIX_LEAN main (2026-08-28) — failed.
2. agentionai FAST head vs kingjones STRIX_LEAN main (2026-08-28) — failed.
3. agentionai FAST head vs agentionai v2 main (2026-08-29, this deployment)
   — failed identically.

The 2.28 GiB head stays on disk; the moment a binary or head appears that
carries the full per-layer norm set, re-enable with
`SPEC_TYPE=draft-mtp,ngram-mod`, `--spec-draft-n-max 3` (author-measured
best: 32.4 t/s, 0.612 acceptance; adaptive p-split measured worse).

## Config files

- Env: `scripts/frank-large.env`
- Serve: `scripts/serve-frank-large.sh`
- Chat template: `scripts/chat-template.jinja` (unchanged, Qwen-Sharp)
- Fork patch: `scripts/patches/qwen4exp-qsa-f32-mask.patch` (re-apply after
  any clean fork rebuild — see Serving profile)

## Swap procedure

```bash
sudo systemctl stop frank-large
# edit scripts/frank-large.env + scripts/serve-frank-large.sh, commit, push
# on Dankclaw: git pull --ff-only && scripts/deploy.sh dankclaw --service frank-large
sudo systemctl start frank-large   # or via deploy.sh
curl http://192.168.0.143:7770/health
```

Model load is slow (87 GiB + 2.3 GiB draft): health returns 503 while
loading; `deploy.sh` waits up to 20 minutes (`LLM_WAIT`). Do not kill it.

## Known issues

1. **ROCmFP4 types require the ROCmFPX fork.** Stock llama.cpp and the CIRU
   runtime reject these files.
2. **Use the `v2/` joined layout, NOT the root `-ple16` split.** This binary's
   qwen4exp loader (qwen4exp.cpp:104-108) unconditionally requires a single
   `per_layer_token_embd.weight` tensor and GGML_ASSERTs "qwen4exp is missing
   the PLE n-gram table" against the per-head split. The root file only loads
   in the LaurentZuijdwijk Vulkan fork (`vulkan-qwen4exp-rocmfpx`). The v2
   file is byte-identical in weights, ~5 KB larger header, loads fully into
   SVM unified memory (no `--ngram-on-disk` needed — kingjones STRIX ran a
   50 GiB joined PLE table the same way).
3. **No MTP draft head loads on build-qwen38.** The qwen4exp loader
   requires a full per-layer `blk.<il>.hc_attn_norm.weight` set; every
   Flash-Next-era head checked (EasiiX Q8_0, agentionai FAST, CIRU) carries
   only `blk.48`. Spec decoding is ngram-mod only. See the MTP section.
4. **`--flash-attn on` (restored 2026-08-29).** fa on aborts graph build
   without the QSA F32-mask patch — see Serving profile (fix + sha256
   vendored there).
5. **`SVM mapping failed` kernel spam** is a documented symptom, not root
   cause — investigate before blaming memory pressure.

## Citations

- [agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF)
- [agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF)
- [Qwen/Qwen3.8-Flash-Next base model](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)
- [charlie12345/ROCmFPX](https://github.com/charlie12345/ROCmFPX) — fork with PR #27742 (qwen4exp architecture) and the ROCmFP4 types
- [llama.cpp PR #27742](https://github.com/ggml-org/llama.cpp/pull/27742) — qwen4exp support (merged upstream)
- [llama.cpp PR #27739](https://github.com/ggml-org/llama.cpp/pull/27739) — MTP draft head support