frank-default-large: re-enable --flash-attn on via QSA F32-mask patch
fa on aborted graph build: build_attn_qsa called ggml_fill on the F16-converted kq mask (GGML_ASSERT F32 at ggml.c:5371). Patch mirrors the MLA sparse path in llm_graph_context::build_attn (F32 base mask for the top-k surgery, F16 cast only at the attention call, gated on cparams.flash_attn). Vendored at scripts/patches/qwen4exp-qsa-f32-mask.patch (sha256 f2f4a486...); the build-qwen38 binary on Dankclaw is rebuilt with it and validated live at CTX_SIZE=32768 (health 200, real completion).
This commit is contained in:
@@ -298,9 +298,10 @@ slot (author quick-start; the fully GPU-resident 87 GiB model + f16 KV +
|
||||
compute buffer OOMs the SVM pool beyond that — verified live: cudaMalloc
|
||||
fails at 131072/262144), b 4096 / ub 4096, t 16 / tb 16, f16 KV cache,
|
||||
`--fit off` (auto-fit segfaults on qwen4exp hyper-connection config),
|
||||
`--flash-attn off` (required: fa on aborts graph build on the QSA indexer
|
||||
path — F32 `ggml_fill` on the F16 kq mask, ggml.c:5371), and reasoning on (Sharp
|
||||
template, budget -1, `--reasoning-format deepseek`; chat kwargs keep
|
||||
`--flash-attn on` (restored 2026-08-29 via the QSA F32-mask patch,
|
||||
`scripts/patches/qwen4exp-qsa-f32-mask.patch` — re-apply inside
|
||||
`/home/dank/GitPackages/ROCmFPX` before any clean rebuild), and reasoning on
|
||||
(Sharp template, budget -1, `--reasoning-format deepseek`; chat kwargs keep
|
||||
`preserve_thinking` + `preserve_reasoning`). Env
|
||||
`scripts/frank-default-large.env`, serve `scripts/serve-frank-large.sh`.
|
||||
|
||||
@@ -309,7 +310,10 @@ model (v2 joined PLE layout), replacing the kingjones STRIX and jcbtc CIRU v1.1
|
||||
deployments. The own-matched MTP draft head does NOT load on build-qwen38 —
|
||||
verified against the v2 main itself (`missing tensor blk.0.hc_attn_norm.weight`),
|
||||
third confirmation (EasiiX Q8_0 + agentionai FAST vs STRIX_LEAN earlier). The
|
||||
head stays on disk for a future compatible binary.
|
||||
head stays on disk for a future compatible binary. Same day, `--flash-attn on`
|
||||
was restored: the QSA F32-mask patch (vendored
|
||||
`scripts/patches/qwen4exp-qsa-f32-mask.patch`) fixes the F16-kq_mask
|
||||
`ggml_fill` abort; the rebuilt fork binary carries it, verified live at 32K.
|
||||
Runbook:
|
||||
[serverAppConfigs-docs/LLM/agentionai-fast-imatrix.md](serverAppConfigs-docs/LLM/agentionai-fast-imatrix.md).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user