chore(skills): promote runbook skills to managed-skills; drop superseded SKILL.md

- 31 new managed skills (anubis/arr/authentik/hermes/freetoken/podman/llm-swap runbooks etc.) now tracked. - Removed superseded skills: omp_harness_config, peon_ping_integration (replaced by user-authored peon-ping-* managed skills), rocmfp4-strix-benchmark-orchestration. - New memory-root skills: niri_compositor_ops, omp_git_workflow, omp_model_config, quadlet-media-migration, homelab-service-triage, hermes-dankclaw-ops, frank-default-large-ops, caddy-auth-dns-playbook.
2026-09-09 16:20:28 -07:00
1 parent 7be39d60dd
commit 9bfa0be85d
@@ -0,0 +1,73 @@
+---
+name: llama-flash-attn-qsa-fix
+description: Restore --flash-attn on for qwen4exp models when QSA indexer path aborts with GGML_ASSERT F32 error
+---
+
+## Problem
+`--flash-attn on` aborts graph build for qwen4exp models:
+```
+GGML_ASSERT(a->type == GGML_TYPE_F32)
+at ggml/src/ggml.c:5371, backtrace through build_attn_qsa
+```
+Root cause: `build_attn_qsa` calls `ggml_fill` on the F16-cast kq mask returned by `get_kq_mask()` when flash attn is enabled, but `ggml_fill` requires F32.
+
+## Solution
+Apply the QSA F32-mask patch that mirrors the MLA sparse path (`llm_graph_context::build_attn`):
+1. Use the F32 base mask (`inp->self_kq_mask`) instead of the F16-cast version for the top-k surgery
+2. Cast to F16 only at the attention call when `flash_attn` is enabled
+
+### Patch hunks (unified diff format)
+```diff
+@@ -560,9 +565,13 @@ ggml_tensor * llama_model_qwen4exp::graph::build_attn_qsa(
+         ggml_build_forward_expand(gf, mctx_cur->cpy_v(ctx0, v_cur, v_idxs, il));
+     }
+-    ggml_tensor * kq_mask = inp->get_kq_mask();
+    // get_kq_mask() returns the F16-converted mask when flash attention is
+    // on, but ggml_fill/set_rows require F32. Mirror the MLA sparse path
+    // (llm_graph_context::build_attn): run the top-k mask surgery on the F32
+    // base mask and cast to the attention type only at the end.
+    ggml_tensor * kq_mask = inp->self_kq_mask;
+     // prepare new kq mask - starts filled with -INFINITY
+     ggml_tensor * kq_mask_all = ggml_fill(ctx0, kq_mask, -INFINITY);
+ 
+@@ -592,9 +601,12 @@ ggml_tensor * llama_model_qwen4exp::graph::build_attn_qsa(
+     ggml_tensor * q = q_cur;
+     ggml_tensor * k = mctx_cur->get_k(ctx0, il);
+     ggml_tensor * v = mctx_cur->get_v(ctx0, il);
+-    ggml_tensor * cur = build_attn_mha(q, k, v, nullptr, kq_mask_top_k, nullptr, nullptr, kq_scale, il);
+    // flash attention consumes an F16 mask; softmax consumes F32 as-is
+    ggml_tensor * kq_mask_attn = inp->cparams.flash_attn ? ggml_cast(ctx0, kq_mask_top_k, GGML_TYPE_F16) : kq_mask_top_k;
+
+    ggml_tensor * cur = build_attn_mha(q, k, v, nullptr, kq_mask_attn, nullptr, nullptr, kq_scale, il);
+     cb(cur, "kqv_out", il);
+     return cur;
+ }
+```
+
+## Procedure
+1. **Apply patch to fork:**
+   ```bash
+   cd /home/dank/GitPackages/ROCmFPX
+   git apply /path/to/qwen4exp-qsa-f32-mask.patch
+   make -j16 llama-server
+   ```
+2. **Validate live (on test port):**
+   ```bash
+   sudo systemctl stop frank-default-large
+   nohup ./build-qwen38/bin/llama-server \n    --model ... --flash-attn on --ctx-size 32768 --fit off -t 16 -tb 16 \n    --port 7999 > /tmp/fa-test.log 2>&1 &
+   sleep 90 && grep 'model loaded' /tmp/fa-test.log && curl http://127.0.0.1:7999/health
+   ```
+3. **Verify completion:** Send a real `/v1/chat/completions` request — if it completes without asserts, the fix works.
+4. **Deploy production:** Commit config (FLASH_ATTN=on), push, pull on target, restart service.
+
+## Verification
+- Model loads without asserts
+- Health endpoint returns 200
+- Real completion succeeds (finish: "stop")
+- Journal clean: no `GGML_ASSERT`, no `cudaMalloc`, no `missing tensor`
+- Flash attention engaged (check server logs for "flash attention" or timing improvement vs off)
+
+## Known Constraints
+- Works only with charlie12345/ROCmFPX build-qwen38 (PR #27742 qwen4exp merged)
+- Does NOT work with CIRU runtime, upstream llama.cpp, or stock binaries
+- ctx-size cap remains 32768 for full-model GPU residency (87 GiB model + KV buffer OOMs beyond)