frank-default-large: re-enable --flash-attn on via QSA F32-mask patch

fa on aborted graph build: build_attn_qsa called ggml_fill on the
F16-converted kq mask (GGML_ASSERT F32 at ggml.c:5371). Patch mirrors the
MLA sparse path in llm_graph_context::build_attn (F32 base mask for the
top-k surgery, F16 cast only at the attention call, gated on
cparams.flash_attn). Vendored at scripts/patches/qwen4exp-qsa-f32-mask.patch
(sha256 f2f4a486...); the build-qwen38 binary on Dankclaw is rebuilt with
it and validated live at CTX_SIZE=32768 (health 200, real completion).
This commit is contained in:
Frank
2026-08-29 23:43:25 -07:00
parent 101b1f8be2
commit 7cb6eeb67a
5 changed files with 70 additions and 21 deletions
+8 -4
View File
@@ -298,9 +298,10 @@ slot (author quick-start; the fully GPU-resident 87 GiB model + f16 KV +
compute buffer OOMs the SVM pool beyond that — verified live: cudaMalloc
fails at 131072/262144), b 4096 / ub 4096, t 16 / tb 16, f16 KV cache,
`--fit off` (auto-fit segfaults on qwen4exp hyper-connection config),
`--flash-attn off` (required: fa on aborts graph build on the QSA indexer
path — F32 `ggml_fill` on the F16 kq mask, ggml.c:5371), and reasoning on (Sharp
template, budget -1, `--reasoning-format deepseek`; chat kwargs keep
`--flash-attn on` (restored 2026-08-29 via the QSA F32-mask patch,
`scripts/patches/qwen4exp-qsa-f32-mask.patch` — re-apply inside
`/home/dank/GitPackages/ROCmFPX` before any clean rebuild), and reasoning on
(Sharp template, budget -1, `--reasoning-format deepseek`; chat kwargs keep
`preserve_thinking` + `preserve_reasoning`). Env
`scripts/frank-default-large.env`, serve `scripts/serve-frank-large.sh`.
@@ -309,7 +310,10 @@ model (v2 joined PLE layout), replacing the kingjones STRIX and jcbtc CIRU v1.1
deployments. The own-matched MTP draft head does NOT load on build-qwen38 —
verified against the v2 main itself (`missing tensor blk.0.hc_attn_norm.weight`),
third confirmation (EasiiX Q8_0 + agentionai FAST vs STRIX_LEAN earlier). The
head stays on disk for a future compatible binary.
head stays on disk for a future compatible binary. Same day, `--flash-attn on`
was restored: the QSA F32-mask patch (vendored
`scripts/patches/qwen4exp-qsa-f32-mask.patch`) fixes the F16-kq_mask
`ggml_fill` abort; the rebuilt fork binary carries it, verified live at 32K.
Runbook:
[serverAppConfigs-docs/LLM/agentionai-fast-imatrix.md](serverAppConfigs-docs/LLM/agentionai-fast-imatrix.md).