serverAppConfigs/docs/LLM/kingjones-strix-rocmfp4.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX on Dankclaw	kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF served by the ROCmFPX llama.cpp fork. 177B params (56B MoE + 51.2B PLE), single 113.47 GiB GGUF. frank-default-large.	llama.cpp rocm dankclaw gfx1151 kingjones rocmfp4 strix-halo qwen4exp	2026-08-27

kingjones Qwen3.8-Flash-Next ROCmFP4 STRIX on Dankclaw

Active since 2026-08-27. Replaced the STRIX_LEAN deployment on the same build. Same GPU memory profile, larger PLE table.

frank-default-large (port 7770) serves Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX from kingjones777 on Hugging Face. A single GGUF file, 113.47 GiB. Text-only (no mmproj, no MTP).

Model Architecture

Base model: Qwen3.8-Flash-Next (177B parameters total).

Component	Params	Details
MoE experts	125.8B	512 experts, 10 used per token
Attention	dense	48 layers (12 attn + 36 SSM)
PLE table	51.2B	per_layer_token_embd, Q8_0
lm head	~0.3%	Q6_K

The PLE (per-layer embedding) n-gram table is the only thing that differs between STRIX, STRIX_LEAN, and FAST tiers. Everything else is byte-identical.

File Details

Field	Value
File	Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf
Size	113.47 GiB
Format	Single GGUF (not sharded)
file_type	105 (Q4_0_ROCMFP4_STRIX)
General	113.47 GiB, 5.51 bpw

Tensor composition:

Tensor group	Type
MoE expert weights (ffn_*_exps, 144)	TYPE_101 (ROCmFP4, 4.251 bpw)
Shared expert (ffn_*_shexp, 144)	TYPE_101
Attention (attn_*, 120)	half TYPE_100, half TYPE_101
PLE (per_layer_token_embd.weight)	Q8_0 — 50.66 GiB
token_embd.weight	Q6_K
output.weight (lm head)	Q6_K
Norms / biases	F32

File Type Reference

Tier	file_type	PLE table	Total size
FAST	TYPE_106	25.34 GiB	87.94 GiB
STRIX_LEAN	TYPE_105 (old STRIX_LEAN)	35.76 GiB	98.49 GiB
STRIX	105 (Q4_0_ROCMFP4_STRIX)	50.66 GiB	113.47 GiB

TYPE_100 = 4.506 bpw, TYPE_101 = 4.251 bpw (measured exactly by solving three files for two unknowns).

Memory

The PLE table never enters GPU VRAM. It streams from the file through the OS page cache.

Context	GTT at load	GTT after prompt	GPU resident
8,192	63.3 GiB	63.6 GiB	~63 GiB
32,768	64.1 GiB	64.8 GiB
65,536	65.1 GiB	66.2 GiB
131,072	67.2 GiB	69.1 GiB	~69 GiB

~63 GiB is the non-PLE mass plus compute buffers. The PLE table does not grow GPU memory regardless of context depth.

Three critical consequences:

1. --no-mmap gets you OOM-killed. It forces the PLE into anonymous memory (neither file-backed nor reclaimable). The cgroup killer takes the process with nothing in the server log — only dmesg shows the evidence.
2. Don't force PLE to CPU with -ot. Measured generation dropped from ~23 to 13.4 tok/s. The kernel streams it better than any offload hack.
3. A bigger tier costs disk, not GPU memory. If you have storage, take STRIX.

Performance

Measured on Ryzen AI MAX+ 395 (gfx1151, ROCm 7.2.4), 49/49 offload. Prompts sized to 85% of context window:

Context	Prompt tokens	pp tok/s	gen tok/s
8,192	6,963	279	22.02
32,768	27,852	264	20.52
65,536	55,705	234	18.23
131,072	111,411	185	15.34

Short prompts (~3,300 tokens): 381 tok/s prompt processing, 22.7 tok/s generation (median of 3 runs).

Serving Profile

Runtime: charlie12345/ROCmFPX llama-server built at /home/dank/GitPackages/ROCmFPX/build-qwen38.

Required flags:

llama-server \
  --model Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf \
  --host 0.0.0.0 --port 7770 --jinja \
  --device ROCm0 --gpu-layers 999 \
  --fit off --flash-attn on \
  --ctx-size 524288 --parallel 2 \
  --batch-size 4096 --ubatch-size 4096 \
  --cache-type-k f16 --cache-type-v f16 \
  --threads 16 --threads-batch 32 \
  --ctx-checkpoints 16 --cache-ram 8192 \
  --cache-prompt --cache-idle-slots --metrics \
  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0 \
  --presence-penalty 1.5 \
  --reasoning on --reasoning-format deepseek --reasoning-budget -1 \
  --chat-template-file chat-template.jinja \
  --chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'

Key choices:

• --fit off — auto-fit code segfaults on this model's qwen4exp hyper-connection/indexer configuration.
• mmap (default) — do NOT use --no-mmap. See memory section.
• --gpu-layers 999 — all layers on SVM-backed GPU pool.
• --ctx-size 524288 — above model's max (131072); llama.cpp clamps internally.

Deployment

Download

# on Dankclaw
TOKEN=$(cat ~/.cache/huggingface/token)
DIR=/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX
mkdir -p "$DIR"
curl -fsSL -C - -o "$DIR/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf" \
  --retry 5 --retry-delay 3 --retry-max-time 7200 \
  -H "Authorization: Bearer $TOKEN" \
  "https://huggingface.co/kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF/resolve/main/Qwen3.8-Flash-Next-Q4_0-ROCmFP4-STRIX.gguf"

The file is 113.47 GiB. Allow ~20-30 minutes for download depending on network.

Swap procedure

1. Stop service: sudo systemctl stop frank-default-large
2. Remove old model dir: rm -rf /home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX_LEAN
3. Start service: sudo systemctl start frank-default-large
4. Verify health: curl http://192.168.0.143:7770/health

Config files

• Env: scripts/frank-default-large.env
• Serve: scripts/serve-frank-large.sh
• Chat template: scripts/chat-template.jinja

Known Issues

1. Patched llama.cpp required. PR #27742 merged into the ROCmFPX fork. Stock builds reject these files (TYPE_100/101 have no number in stock ggml).
2. Single-file payload. Not sharded. Use hf download or a resumable curl. Browser download unreliable for 113 GiB.
3. Model load is slow. A few minutes cold for the 113 GiB file with a fresh KV cache. Health returns 503 while loading. Do not kill the process. deploy.sh waits up to 20 minutes.
4. Kernel spam about amdgpu: SVM mapping failed is a known documented symptom, not root cause. Investigate other causes before assuming memory pressure.

Citations

• kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF
• Qwen/Qwen3.8-Flash-Next base model
• charlie12345/ROCmFPX — fork with PR #27742 (qwen4exp architecture)