frank-default-large: re-enable --flash-attn on via QSA F32-mask patch

fa on aborted graph build: build_attn_qsa called ggml_fill on the
F16-converted kq mask (GGML_ASSERT F32 at ggml.c:5371). Patch mirrors the
MLA sparse path in llm_graph_context::build_attn (F32 base mask for the
top-k surgery, F16 cast only at the attention call, gated on
cparams.flash_attn). Vendored at scripts/patches/qwen4exp-qsa-f32-mask.patch
(sha256 f2f4a486...); the build-qwen38 binary on Dankclaw is rebuilt with
it and validated live at CTX_SIZE=32768 (health 200, real completion).
This commit is contained in:
Frank committed 2026-08-29 23:43:25 -07:00
1 parent 101b1f8be2
commit 7cb6eeb67a
5 files changed
+70 -21

No files matched your search

+12 -8
View File
@@ -10,12 +10,16 @@
#
# Model: Qwen3.8-Flash-Next base (qwen4exp MoE: 512 experts / 10 used,
# 48 layers = 12 attention + 36 SSM, QSA indexer, hyper-connections, PLE
# KNOWN CONSTRAINTS on the charlie12345/ROCmFPX build-qwen38 runtime.
# 1. FA must stay OFF. --flash-attn on aborts graph build: the QSA
# indexer path calls ggml_fill on the F16-cast kq mask (ggml_fill
# asserts GGML_TYPE_F32 at ggml.c:5371). The v2 main loads and serves
# fine with --flash-attn off (verified live: model loaded, health 200,
# real completion).
# 1. Flash attention is ON. fa on used to abort graph build: the QSA
# indexer path called ggml_fill on the F16-cast kq mask (ggml_fill
# asserts GGML_TYPE_F32 at ggml.c:5371). Fixed in the fork source by
# the QSA F32-mask patch (scripts/patches/qwen4exp-qsa-f32-mask.patch,
# sha256 f2f4a486e21cfad80bfa42683281d18e1cba22fa43c5e0e2aa0cfdbe141e925f)
# and the rebuilt binary at LLAMA_BIN already carries the fix. A clean
# rebuild must re-apply: git apply scripts/patches/qwen4exp-qsa-f32-mask.patch
# inside /home/dank/GitPackages/ROCmFPX before make. Verified live:
# --flash-attn on + v2 main at CTX_SIZE=32768 loads, health 200, real
# completion.
# 2. The MTP draft head does NOT load on this binary. The qwen4exp loader
# requires blk.0.hc_attn_norm.weight per layer; the agentionai FAST
# head carries only blk.48 (verified 3x: EasiiX Q8_0 + FAST against
@@ -54,7 +58,7 @@ ALIAS=frank-default-large
BIND_ADDR=0.0.0.0
LISTEN_PORT=7770
# Serving: 131072-token context on ONE slot (parallel 1), batch 4096 /
# Serving: 32768-token context on ONE slot (parallel 1), batch 4096 /
# ubatch 4096, 16+16 threads, f16 KV, 16 context checkpoints, 8 GiB
# prompt cache. --fit off: auto-fit segfaults on this model's
# hyper-connection/indexer config; -ngl 999 puts every layer on the
@@ -81,7 +85,7 @@ SLOT_SAVE_PATH=/home/dank/serverAppConfigs/scripts/slot-state
SPLIT_MODE=none
FIT=off
NGL=999
FLASH_ATTN=off
FLASH_ATTN=on
# Sampling (card instruct table, non-thinking).
TEMPERATURE=0.7