frank-default-large: swap to agentionai Qwen3.8-Flash-Next ROCmFP4-FAST imatrix + own MTP draft

Model: agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF (87.06 GiB,
Q4_0_ROCMFP4_FAST imatrix-calibrated, per-head PLE table inside the GGUF —
fully VRAM-resident). Draft: agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST
(2.28 GiB, matched ROCmFP4-FAST quant, ~66% acceptance at n-max 3).
Vision tower (f16) loaded via mmproj/.

Runtime: charlie12345/ROCmFPX build-qwen38 (HIP, ROCm 7.2.4, gfx1151) —
the only binary with the ROCmFP4 FAST types. Replaces the jcbtc CIRU v1.1
deployment (ciru-ai runtime).

Profile: card sampling table (temp 0.7, top_p 0.8, top_k 20, min_p 0.0,
presence 1.5) mixed with the validated build-qwen38 serving profile:
262144 ctx x 1 slot, b 4096 / ub 4096, t 16 / tb 16, f16 KV, fa on,
cont-batching, prompt cache + 8 GiB cache-ram + idle slots, 16 ctx
checkpoints (no --checkpoint-min-step — unsupported on this binary),
--fit off (auto-fit segfaults on qwen4exp hyper-connection config),
--spec-type draft-mtp,ngram-mod with n-max 3 (adaptive p-split measured
worse per author card), p-min 0.75. Qwen-Sharp chat template kept;
reasoning stays in DeepSeek format.

The 2026-08-28 failure of this draft head on the kingjones STRIX_LEAN
model was a cross-vendor pairing problem: the head carries
blk.48.hc_attn_norm.weight (verified in-GGUF, same shape as the working
CIRU draft). Docs: new runbook agentionai-fast-imatrix, index updates.

scripts/frank-default-large.env

@@ -1,45 +1,48 @@
-# Tunables for frank-default-large (jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 on
-# the ciru-ai Qwen3.8-Flash-CIRU-STRIX-IU4 llama.cpp runtime, tag v1.1,
-# Dankclaw, Strix Halo gfx1151 unified memory).
+# Tunables for frank-default-large (agentionai/Qwen3.8-Flash-Next-ROCmFP4-
+# FAST-imatrix-GGUF on the charlie12345/ROCmFPX llama.cpp runtime,
+# build-qwen38, Dankclaw, Strix Halo gfx1151 unified memory).
 #
-# Runtime: full llama.cpp fork at
-# /home/dank/GitPackages/ciru-Qwen3.8-Flash-CIRU-STRIX-IU4/build-gfx1151,
-# built with the vendor build-linux-amd.sh flag set (HIP on, HIP_NO_VMM on,
-# HIP_GRAPHS on, HIP_MMQ_MFMA on, CUDA/VULKAN/SYCL off, gfx1151) plus
-# GGML_CUDA_FA + GGML_CUDA_FA_ALL_QUANTS. The IU4 WMMA expert path and the
-# CIRUPLE1 PLE pager read the GGML_* env knobs exported by the serve script.
-# Replaced the kingjones STRIX_LEAN (ROCmFPX build-qwen38) deployment.
+# Runtime: charlie12345/ROCmFPX llama.cpp at
+# /home/dank/GitPackages/ROCmFPX/build-qwen38 (HIP, ROCm 7.2.4, gfx1151,
+# PR #27742 qwen4exp merged). The ROCmFP4 quant types are hand-ported from
+# ciru-ai/ROCmFPX; upstream llama.cpp cannot load this GGUF. Replaced the
+# jcbtc CIRU v1.1 (ciru-ai runtime) and kingjones STRIX_LEAN deployments.
 #
-# Model: Qwen3.8-Flash-Next base (56B MoE, 512 experts / 10 used, 48 layers
-# = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE at layer 1)
-# in the CIRU quality-first mixed quant: 144 routed-expert tensors at Q4_1
-# on the native IU4 WMMA path, protected tensors at Q5_K/Q5_1/Q8_0/BF16/F32.
-# Main GGUF ~79 GB + MTP Q8_0 draft ~4 GB + exact-FP8 PLE sidecar ~52 GB
-# paged from disk (never in VRAM; 4 GiB decoded-page cache via
-# --ple-cache-mib). Vendor-validated on the same silicon (gfx1151).
+# Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
+# 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
+# at layer 1) quantized Q4_0_ROCMFP4_FAST imatrix-calibrated, 87.06 GiB,
+# 4.23 bpw. The n-gram PLE table is split per head inside the single GGUF
+# (fully VRAM-resident — no ple sidecar, no --ngram-on-disk). Vision tower
+# (f16, lossless) ships as mmproj/. The model's own MTP head (2.28 GiB,
+# matched ROCmFP4-FAST quant, ~66% draft acceptance) drives spec decoding.
 #
-# The vendor production profile is applied verbatim on this first deployment:
-# -ngl all, -sm none, --fit off, ctx 262144 x 1 slot, b 2048 / ub 512,
-# t 8 / tb 8, f16 KV, fa on, cont-batching, prompt cache + 8 GiB cache-ram
-# + idle-slot cache, 32 ctx checkpoints with min step 8192, MTP depth 3
-# with p-split 0.10, GGML_CUDA_Q41_MOE_FORCE_J=32,
-# GGML_QWEN4EXP_PLE_WORKERS=16, GGML_QWEN4EXP_PLE_STRICT_SHA=0 (checksums
-# were verified with sha256sum -c at download time), ROCBLAS_USE_HIPBLASLT=1.
-# The Qwen-Sharp chat template is kept on purpose (chat-template.jinja): the
-# service keeps emitting reasoning in DeepSeek format for its clients.
+# Profile: card sampling table (temp 0.7, top_p 0.80, top_k 20, min_p 0.0,
+# presence 1.5 — non-thinking instruct) mixed with the validated
+# build-qwen38 serving profile: ctx 262144 x 1 slot, b 4096 / ub 4096,
+# t 16 / tb 16 (threads-batch 16 measured best for prefill), f16 KV, fa on,
+# cont-batching, prompt cache + 8 GiB cache-ram + idle-slot cache, 16 ctx
+# checkpoints, --fit off (auto-fit segfaults on qwen4exp
+# hyper-connection/indexer config). MTP depth 3 (n-max 3 measured best on
+# this head; adaptive p-split measured worse, left off). The Qwen-Sharp
+# chat template is kept on purpose (chat-template.jinja): the service keeps
+# emitting reasoning in DeepSeek format for its clients.
 #
 # WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The
 # ROCm HIP runtime reads those bare env vars (see the 2026-08-13 TEMP
 # incident). llama.cpp reads --host/--port/--temp as CLI flags, never as
 # env vars.
 
-# CIRU v1.1 runtime binary (vendor build script, plus FA_ALL_QUANTS).
-LLAMA_BIN=/home/dank/GitPackages/ciru-Qwen3.8-Flash-CIRU-STRIX-IU4/build-gfx1151/bin/llama-server
+# ROCmFPX (build-qwen38) runtime binary.
+LLAMA_BIN=/home/dank/GitPackages/ROCmFPX/build-qwen38/bin/llama-server
 
-# Model package from https://huggingface.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4
-# Layout: main GGUF + mtp/ Q8_0 draft + ple/ CIRUPLE1 sidecar (mandatory).
-MODEL_DIR=/home/dank/models/ciru-Qwen3.8-Flash-CIRU-STRIX-IU4
-MODEL_FILE=Qwen3.8-Flash-CIRU-STRIX-IU4.gguf
+# Model package from
+# https://huggingface.co/agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF
+# Layout: single main GGUF (per-head PLE table, fully VRAM-resident) +
+# mmproj/ vision tower. The MTP draft head lives in
+# /home/dank/models/mtp-Qwen3.8-Flash-Next-FAST (already on the host).
+MODEL_DIR=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST
+MODEL_FILE=Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2-ple16.gguf
+MMPROJ_PATH=/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf
 
 # Qwen-Sharp chat template (tracked in the repo, staged on the host). The
 # Sharp template drives thinking on by default (medium effort); the server
@@ -53,28 +56,27 @@ ALIAS=frank-default-large
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7770
 
-# Vendor production profile, verbatim: 262144-token context on ONE slot
-# (parallel 1), batch 2048 / ubatch 512, 8+8 threads, f16 KV, 32 context
-# checkpoints with min step 8192, 8 GiB prompt cache, 4 GiB PLE page cache.
-# --fit off: auto-fit segfaults on this model's hyper-connection/indexer
-# config; -ngl all puts every layer on the SVM-backed GPU pool.
+# Serving: 262144-token context on ONE slot (parallel 1), batch 4096 /
+# ubatch 4096, 16+16 threads, f16 KV, 16 context checkpoints, 8 GiB
+# prompt cache. --fit off: auto-fit segfaults on this model's
+# hyper-connection/indexer config; -ngl 999 puts every layer on the
+# SVM-backed GPU pool.
 CTX_SIZE=262144
 PARALLEL=1
-BATCH_SIZE=2048
-UBATCH_SIZE=512
-NUM_THREADS=8
-THREAD_BATCH=8
+BATCH_SIZE=4096
+UBATCH_SIZE=4096
+NUM_THREADS=16
+THREAD_BATCH=16
 CACHE_TYPE_K=f16
 CACHE_TYPE_V=f16
-CTX_CHECKPOINTS=32
-CHECKPOINT_MIN_STEP=8192
+CTX_CHECKPOINTS=16
 CACHE_RAM=8192
-PLE_CACHE_MIB=4096
 SLOT_SAVE_PATH=/home/dank/serverAppConfigs/scripts/slot-state
 SPLIT_MODE=none
 FIT=off
+NGL=999
 
-# Sampling (Sharp default; vendor API examples agree for non-thinking mode).
+# Sampling (card instruct table, non-thinking).
 TEMPERATURE=0.7
 TOP_K=20
 TOP_P=0.8
@@ -93,26 +95,18 @@ HSA_OVERRIDE_GFX_VERSION=11.5.1
 
 VERBOSITY=4
 
-# Speculative decoding: MTP draft head (Q8_0) at depth 3, p-split 0.10 —
-# the vendor headline profile. ngram-mod stacking is a later experiment;
-# SPEC_TYPE=none disables spec decoding entirely.
-DRAFT_MODEL_DIR=/home/dank/models/ciru-Qwen3.8-Flash-CIRU-STRIX-IU4/mtp
-DRAFT_MODEL_FILE=Qwen3.8-Flash-CIRU-STRIX-IU4-MTP-Q8_0.gguf
-SPEC_TYPE=draft-mtp
+# Speculative decoding: the model's own MTP head (agentionai
+# Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF, 2.28 GiB, ROCmFP4-FAST quant
+# matched to the target) at depth 3 stacked on ngram-mod. n-max 3 measured
+# best (32.4 t/s, 0.612 acceptance); adaptive p-split measured worse, so it
+# stays off (p-min stays at the 0.75 default). Set SPEC_TYPE=none to disable.
+DRAFT_MODEL_DIR=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST
+DRAFT_MODEL_FILE=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf
+SPEC_TYPE=draft-mtp,ngram-mod
 SPEC_DRAFT_N_MAX=3
 SPEC_DRAFT_N_MIN=0
-SPEC_DRAFT_P_MIN=0.0
-SPEC_DRAFT_P_SPLIT=0.10
-SPEC_DRAFT_NGL=all
+SPEC_DRAFT_P_MIN=0.75
+SPEC_DRAFT_NGL=999
 SPEC_DRAFT_DEVICE=ROCm0
-SPEC_DRAFT_TYPE_K=q8_0
-SPEC_DRAFT_TYPE_V=q8_0
-
-# Vendor runtime env knobs (read by the hipified mmq + PLE pager kernels).
-# GGML_CUDA_Q41_MOE_FORCE_J forces the Q4_1 MoE WMMA tile width; the PLE
-# pager knobs control the CIRUPLE1 page-in workers and skip re-hashing the
-# 52 GB payload at every launch (checksums were verified once, at download).
-GGML_CUDA_Q41_MOE_FORCE_J=32
-GGML_QWEN4EXP_PLE_WORKERS=16
-GGML_QWEN4EXP_PLE_STRICT_SHA=0
-ROCBLAS_USE_HIPBLASLT=1
+SPEC_DRAFT_TYPE_K=f16
+SPEC_DRAFT_TYPE_V=f16