Document Q8 AGENT set_rows serving blocker

This commit is contained in:
Frank committed 2026-08-25 12:16:47 -07:00
1 parent 54b2d84613
commit 6c9e71efd2
3 files changed
+72 -6

No files matched your search

+10
View File
@@ -196,6 +196,16 @@ budget (raised 2026-08-25). Env `scripts/frank-default-large.env`, serve
`scripts/serve-frank-large.sh`, runbook
[docs/LLM/kingjones-q8agent-rocmfpx.md](docs/LLM/kingjones-q8agent-rocmfpx.md).
**Serving status (2026-08-25):** the model loads but crashes in
`ggml_compute_forward_set_rows` (NULL dispatch in `libggml-cpu.so.0`) before it
can serve. Tested on all five available ROCmFPX builds (build-kairic v81,
build-promptforge v3, build-rocmi4-w4a4 v257, build-strix-rocmfp4 v209) and the
all-CPU path; all crash. Not GPU-specific, not MTP-specific, not fixed by
`LLAMA_SET_ROWS=0`. Fix is a rebuild of charlie12345/ROCmFPX with a working
`set_rows`. The proven serving context is 131072 at `gl=45` (262144 OOMs on
device-local at `gl=999`). Config goals met: 114 GiB TTM/GTT, reasoning on,
`preserve_thinking`/`preserve_reasoning` on.
### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, prior v3, 2026-08-18 to 2026-08-25)
The v4 runtime serves `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (`eemin` namespace,