frank-default-large: ngram-mod only — both MTP draft heads incompatible
Both the EasiiX Q8_0 and agentionai ROCmFP4-FAST MTP draft heads fail with 'missing tensor blk.0.hc_attn_norm.weight' against the STRIX_LEAN Uncensored model. The qwen4exp binary expects this hyper-connection attn norm tensor which neither draft GGUF carries. Using ngram-mod only until a compatible MTP head is found.
This commit is contained in:
1 parent
b1f30e5235
commit
6d4a96a423
2 files changed
+12
-11
No files matched your search
@@ -21,11 +21,11 @@
|
||||
# hyper-connection/indexer configuration; -ngl 999 places all layers on
|
||||
# the GPU (SVM-backed).
|
||||
#
|
||||
# MTP draft head: agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF
|
||||
# (ROCmFP4-FAST, 2.28 GiB). Speculative decoding via --spec-type
|
||||
# draft-mtp,ngram-mod (n_max 3, p_min 0.75). The earlier EasiiX Q8_0
|
||||
# draft was incompatible (missing blk.0.hc_attn_norm.weight). Set
|
||||
# SPEC_TYPE=none in the env to disable spec decoding.
|
||||
# MTP draft head: both attempted heads (EasiiX Q8_0 and agentionai
|
||||
# ROCmFP4-FAST) are INCOMPATIBLE with the STRIX_LEAN model — the
|
||||
# qwen4exp binary expects blk.0.hc_attn_norm.weight which neither
|
||||
# draft GGUF carries. Using ngram-mod only. Set SPEC_TYPE=none in
|
||||
# the env to disable spec decoding.
|
||||
#
|
||||
# The Qwen-Sharp chat template (scripts/chat-template.jinja) is applied at
|
||||
# runtime so the model emits reasoning in DeepSeek format; the reasoning
|
||||
@@ -69,7 +69,7 @@ fi
|
||||
: "${REASONING:=on}"
|
||||
: "${REASONING_FORMAT:=deepseek}"
|
||||
: "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
|
||||
: "${SPEC_TYPE:=draft-mtp,ngram-mod}"
|
||||
: "${SPEC_TYPE:=ngram-mod}"
|
||||
: "${DRAFT_MODEL_DIR:=$MODEL_DIR}"
|
||||
: "${DRAFT_MODEL_FILE:=Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf}"
|
||||
: "${SPEC_DRAFT_N_MAX:=3}"
|
||||
|
||||
Reference in new issue
Block a user