dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-default-large: ngram-mod only — both MTP draft heads incompatible

Browse Source 
Both the EasiiX Q8_0 and agentionai ROCmFP4-FAST MTP draft heads fail
with 'missing tensor blk.0.hc_attn_norm.weight' against the STRIX_LEAN
Uncensored model. The qwen4exp binary expects this hyper-connection
attn norm tensor which neither draft GGUF carries. Using ngram-mod
only until a compatible MTP head is found.
This commit is contained in:
Frank committed 2026-08-28 01:11:23 -07:00
1 parent b1f30e5235
commit 6d4a96a423
2 files changed
+12 -11
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

scripts/frank-default-large.env
+6 -5
View File
@@ -87,13 +87,14 @@ HSA_OVERRIDE_GFX_VERSION=11.5.1
VERBOSITY=4 VERBOSITY=4
# MTP draft head (speculative decoding). agentionai # MTP draft head (speculative decoding) — BOTH attempted heads are
# Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF, 2.28 GiB, ROCmFP4-FAST # INCOMPATIBLE with the STRIX_LEAN Uncensored model (missing tensor
# quant matched to the target. n_max 3 per the HF card (n_max 4 # blk.0.hc_attn_norm.weight): the EasiiX Q8_0 and the agentionai
# measured worse). Set SPEC_TYPE=none to disable spec decoding. # ROCmFP4-FAST. Using ngram-mod only until a compatible MTP head
# is found. Set SPEC_TYPE=none to disable spec decoding entirely.
DRAFT_MODEL_DIR=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST DRAFT_MODEL_DIR=/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST
DRAFT_MODEL_FILE=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf DRAFT_MODEL_FILE=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf
SPEC_TYPE=draft-mtp,ngram-mod SPEC_TYPE=ngram-mod
SPEC_DRAFT_N_MAX=3 SPEC_DRAFT_N_MAX=3
SPEC_DRAFT_N_MIN=0 SPEC_DRAFT_N_MIN=0
SPEC_DRAFT_P_MIN=0.75 SPEC_DRAFT_P_MIN=0.75
scripts/serve-frank-large.sh
+6 -6
View File
@@ -21,11 +21,11 @@
# hyper-connection/indexer configuration; -ngl 999 places all layers on # hyper-connection/indexer configuration; -ngl 999 places all layers on
# the GPU (SVM-backed). # the GPU (SVM-backed).
# #
# MTP draft head: agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF # MTP draft head: both attempted heads (EasiiX Q8_0 and agentionai
# (ROCmFP4-FAST, 2.28 GiB). Speculative decoding via --spec-type # ROCmFP4-FAST) are INCOMPATIBLE with the STRIX_LEAN model — the
# draft-mtp,ngram-mod (n_max 3, p_min 0.75). The earlier EasiiX Q8_0 # qwen4exp binary expects blk.0.hc_attn_norm.weight which neither
# draft was incompatible (missing blk.0.hc_attn_norm.weight). Set # draft GGUF carries. Using ngram-mod only. Set SPEC_TYPE=none in
# SPEC_TYPE=none in the env to disable spec decoding. # the env to disable spec decoding.
# #
# The Qwen-Sharp chat template (scripts/chat-template.jinja) is applied at # The Qwen-Sharp chat template (scripts/chat-template.jinja) is applied at
# runtime so the model emits reasoning in DeepSeek format; the reasoning # runtime so the model emits reasoning in DeepSeek format; the reasoning
@@ -69,7 +69,7 @@ fi
: "${REASONING:=on}" : "${REASONING:=on}"
: "${REASONING_FORMAT:=deepseek}" : "${REASONING_FORMAT:=deepseek}"
: "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}" : "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
: "${SPEC_TYPE:=draft-mtp,ngram-mod}" : "${SPEC_TYPE:=ngram-mod}"
: "${DRAFT_MODEL_DIR:=$MODEL_DIR}" : "${DRAFT_MODEL_DIR:=$MODEL_DIR}"
: "${DRAFT_MODEL_FILE:=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf}" : "${DRAFT_MODEL_FILE:=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf}"
: "${SPEC_DRAFT_N_MAX:=3}" : "${SPEC_DRAFT_N_MAX:=3}"