serverAppConfigs/serverAppConfigs-docs/LLM/kingjones-strix-rocmfp4.md

---
type: reference
title: kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX on Dankclaw
description: kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF served by the ROCmFPX llama.cpp fork. 177B params (56B MoE + 51.2B PLE), single 113.47 GiB GGUF. frank-default-large.
tags:
  - llama.cpp
  - rocm
  - dankclaw
  - gfx1151
  - kingjones
  - rocmfp4
  - strix-halo
  - qwen4exp
timestamp: 2026-08-27
---

# kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX on Dankclaw

> **Active since 2026-08-27.** Replaced the STRIX_LEAN deployment on the same
> build. Same GPU memory profile, larger PLE table.

`frank-default-large` (port 7770) serves
`Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX` from `kingjones777` on Hugging Face.
A single GGUF file, 113.47 GiB. Text-only (no mmproj). MTP draft head
loaded as a separate GGUF sidecar (see MTP Draft Head below).

## Model Architecture

Base model: Qwen3.8-Flash-Next (177B parameters total).

| Component | Params | Details |
|-----------|--------|---------|
| MoE experts | 125.8B | 512 experts, 10 used per token |
| Attention | dense | 48 layers (12 attn + 36 SSM) |
| PLE table | 51.2B | `per_layer_token_embd`, Q8_0 |
| lm head | ~0.3% | Q6_K |

The PLE (per-layer embedding) n-gram table is the only thing that differs
between STRIX, STRIX_LEAN, and FAST tiers. Everything else is byte-identical.

## File Details

| Field | Value |
|-------|-------|
| File | `Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf` |
| Size | 113.47 GiB |
| Format | Single GGUF (not sharded) |
| file_type | 105 (`Q4_0_ROCMFP4_STRIX`) |
| General | 113.47 GiB, 5.51 bpw |

Tensor composition:

| Tensor group | Type |
|---|---|
| MoE expert weights (`ffn_*_exps`, 144) | TYPE_101 (ROCmFP4, 4.251 bpw) |
| Shared expert (`ffn_*_shexp`, 144) | TYPE_101 |
| Attention (`attn_*`, 120) | half TYPE_100, half TYPE_101 |
| PLE (`per_layer_token_embd.weight`) | Q8_0 — 50.66 GiB |
| `token_embd.weight` | Q6_K |
| `output.weight` (lm head) | Q6_K |
| Norms / biases | F32 |

## File Type Reference

| Tier | file_type | PLE table | Total size |
|------|-----------|-----------|------------|
| FAST | TYPE_106 | 25.34 GiB | 87.94 GiB |
| STRIX_LEAN | TYPE_105 (old STRIX_LEAN) | 35.76 GiB | 98.49 GiB |
| **STRIX** | **105 (`Q4_0_ROCMFP4_STRIX`)** | **50.66 GiB** | **113.47 GiB** |

TYPE_100 = 4.506 bpw, TYPE_101 = 4.251 bpw (measured exactly by solving
three files for two unknowns).

## Memory

The PLE table never enters GPU VRAM. It streams from the file through the OS
page cache.

| Context | GTT at load | GTT after prompt | GPU resident |
|---------|-------------|-------------------|-------------|
| 8,192 | 63.3 GiB | 63.6 GiB | ~63 GiB |
| 32,768 | 64.1 GiB | 64.8 GiB | |
| 65,536 | 65.1 GiB | 66.2 GiB | |
| 131,072 | 67.2 GiB | 69.1 GiB | ~69 GiB |

~63 GiB is the non-PLE mass plus compute buffers. **The PLE table does not
grow GPU memory regardless of context depth.**

Three critical consequences:

1. **`--no-mmap` gets you OOM-killed.** It forces the PLE into anonymous memory
   (neither file-backed nor reclaimable). The cgroup killer takes the process
   with nothing in the server log — only `dmesg` shows the evidence.
2. **Don't force PLE to CPU with `-ot`.** Measured generation dropped from
   ~23 to 13.4 tok/s. The kernel streams it better than any offload hack.
3. **A bigger tier costs disk, not GPU memory.** If you have storage, take
   STRIX.

## Performance

Measured on Ryzen AI MAX+ 395 (gfx1151, ROCm 7.2.4), 49/49 offload. Prompts
sized to 85% of context window:

| Context | Prompt tokens | pp tok/s | gen tok/s |
|---------|--------------|----------|-----------|
| 8,192 | 6,963 | 279 | **22.02** |
| 32,768 | 27,852 | 264 | **20.52** |
| 65,536 | 55,705 | 234 | **18.23** |
| 131,072 | 111,411 | 185 | **15.34** |

Short prompts (~3,300 tokens): 381 tok/s prompt processing, 22.7 tok/s
generation (median of 3 runs).
## Benchmark: threads-batch tuning

Sweeped `--threads-batch` (16, 32, 48, 64, 80, 96, 128) with a 64K-token
prompt via HTTP API (`max_tokens=1`). Fresh server start per value.

| threads-batch | prefill tok/s | wall time |
|---------------|--------------|-----------|
| **16** | **373.5** | 62.9s |
| 32 | 354.4 | 66.3s |
| 48 | 344.1 | 68.3s |
| 64 | 344.7 | 68.2s |
| 80 | 344.4 | 68.2s |
| 96 | 357.1 | 65.8s |
| 128 | 321.1 | 73.2s |

**Optimal: `--threads-batch 16`** (373.5 tok/s). Lower thread batching wins
for prefill on this model -- fewer threads compete for GPU resources during
the prompt processing phase. The default of 32 is 5.1% slower. Above 64,
performance plateaus then declines (128 is 14% worse than optimal).

## Benchmark: decode throughput

Sweeped `--threads-batch` (8, 16, 32) with a ~23.5K-token code prompt and
256 generated tokens via HTTP API. Fresh server start per value.

| threads-batch | gen tokens | wall time | gen tok/s |
|---------------|-----------|-----------|-----------|
| 16 | 256 | 74.8s | 3.4 |
| 32 | 256 | 75.0s | 3.4 |
| 8 | 256 | 76.6s | 3.3 |

**Decode throughput is essentially flat across all tested values.** TB=16 and
TB=32 are identical at 3.4 tok/s; TB=8 is marginally slower at 3.3 tok/s.
Combined with the 5.1% prefill advantage at lower thread counts, `--threads-batch
16` is the optimal setting for both phases.

## Serving Profile

Runtime: charlie12345/ROCmFPX `llama-server` built at
`/home/dank/GitPackages/ROCmFPX/build-qwen38`.

Required flags:

```
llama-server \
  --model Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf \
  --host 0.0.0.0 --port 7770 --jinja \
  --device ROCm0 --gpu-layers 999 \
  --fit off --flash-attn on \
  --ctx-size 524288 --parallel 2 \
  --batch-size 4096 --ubatch-size 4096 \
  --cache-type-k f16 --cache-type-v f16 \
  --threads 16 --threads-batch 16 \
  --ctx-checkpoints 16 --cache-ram 8192 \
  --cache-prompt --cache-idle-slots --metrics \
  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0 \
  --presence-penalty 1.5 \
  --reasoning on --reasoning-format deepseek --reasoning-budget -1 \
  --chat-template-file chat-template.jinja \
  --chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}' \
  --spec-type draft-mtp,ngram-mod \
  --model-draft mtp-Qwen3.8-Flash-Next-Q8_0.gguf \
  --spec-draft-n-max 4 --spec-draft-p-min 0.75 \
  --spec-draft-ngl 999 --spec-draft-device ROCm0 \
  --spec-draft-type-k f16 --spec-draft-type-v f16
```

Key choices:

- **`--fit off`** — auto-fit code segfaults on this model's qwen4exp
  hyper-connection/indexer configuration.
- **mmap (default)** — do NOT use `--no-mmap`. See memory section.
- **`--gpu-layers 999`** — all layers on SVM-backed GPU pool.
- **`--ctx-size 524288`** — above model's max (131072); llama.cpp clamps
  internally.
- **`--spec-type draft-mtp,ngram-mod`** — MTP draft head + ngram-mod
  fallback. The `ngram-mod` combo is recommended by the HF card; the
  prior dense 27B measurement showed ngram under MTP hurt by 4-5 t/s
  ([[kingjones-strix-mtp]]). Benchmark both before committing.
- **`--spec-draft-p-min 0.75`** — confidence gate. Without it, low-acceptance
  prose loses more to rejected drafts than it gains (per the HF card).

## MTP Draft Head

The MTP (multi-token-prediction) draft head from
[EasiiX/Qwen3.8-Flash-Next-MTP-Strix-Halo-GGUF](https://huggingface.co/EasiiX/Qwen3.8-Flash-Next-MTP-Strix-Halo-GGUF)
(Q8_0, 4.1 GB). Architecture `qwen4exp`, matching the main model. Loaded
as a separate GGUF sidecar via `--model-draft`.

| Field | Value |
|-------|-------|
| File | `mtp-Qwen3.8-Flash-Next-Q8_0.gguf` |
| Size | 4.1 GB (4,137,429,088 B) |
| Quant | Q8_0 |
| Architecture | `qwen4exp` |
| Source | EasiiX/Qwen3.8-Flash-Next-MTP-Strix-Halo-GGUF |

**Measured (HF card, gfx1151, UD-IQ3_XXS target, temp 0):** decode +50% on
code (23.5 → 35.7 t/s, ~85% acceptance), +49% at 156K context depth.
Speculative decoding is lossless at temp 0. Our service runs at temp 0.7,
so expect lower acceptance; the `p_min 0.75` gate compensates.

**Caveats:**

- The MTP head is from the original `Qwen/Qwen3.8-Flash-Next`; our model is
  the "Uncensored" (abliterated) variant. Abliteration modifies a small
  number of weight vectors, not the full model — acceptance may be
  marginally lower.
- The HF card's tested fork is `Aristo94/EngramHalo.cpp`, not our
  `charlie12345/ROCmFPX`. Both have PRs #27739 (MTP) + #27742 (qwen4exp)
  merged, so the MTP mechanism is the same upstream code. If the draft
  load crashes, check for Strix-Halo kernel patch differences first.
- Set `SPEC_TYPE=none` in `scripts/frank-default-large.env` to disable.

## Deployment

### Download

```bash
# on Dankclaw
TOKEN=$(cat ~/.cache/huggingface/token)
DIR=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX
mkdir -p "$DIR"
curl -fsSL -C - -o "$DIR/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf" \
  --retry 5 --retry-delay 3 --retry-max-time 7200 \
  -H "Authorization: Bearer $TOKEN" \
  "https://huggingface.co/kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF/resolve/main/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf"
```

The file is 113.47 GiB. Allow ~20-30 minutes for download depending on network.

### Swap procedure

1. Stop service: `sudo systemctl stop frank-default-large`
2. Remove old model dir: `rm -rf /home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN`
3. Start service: `sudo systemctl start frank-default-large`
4. Verify health: `curl http://192.168.0.143:7770/health`

### Config files

- Env: `scripts/frank-default-large.env`
- Serve: `scripts/serve-frank-large.sh`
- Chat template: `scripts/chat-template.jinja`

## Known Issues

1. **Patched llama.cpp required.** PR #27742 merged into the ROCmFPX fork.
   Stock builds reject these files (TYPE_100/101 have no number in stock ggml).
2. **Single-file payload.** Not sharded. Use `hf download` or a resumable
   curl. Browser download unreliable for 113 GiB.
3. **Model load is slow.** A few minutes cold for the 113 GiB file with a
   fresh KV cache. Health returns 503 while loading. Do not kill the process.
   `deploy.sh` waits up to 20 minutes.
4. **Kernel spam about `amdgpu: SVM mapping failed`** is a known documented
   symptom, not root cause. Investigate other causes before assuming memory
   pressure.

## Citations

- [kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF](https://huggingface.co/kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF)
- [Qwen/Qwen3.8-Flash-Next base model](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)
- [charlie12345/ROCmFPX](https://github.com/charlie12345/ROCmFPX) — fork with PR #27742 (qwen4exp architecture)
- [EasiiX/Qwen3.8-Flash-Next-MTP-Strix-Halo-GGUF](https://huggingface.co/EasiiX/Qwen3.8-Flash-Next-MTP-Strix-Halo-GGUF) — MTP draft head sidecar
- [llama.cpp PR #27739](https://github.com/ggml-org/llama.cpp/pull/27739) — MTP draft head support