Document Q8 AGENT set_rows serving blocker
This commit is contained in:
1 parent
54b2d84613
commit
6c9e71efd2
3 files changed
+72
-6
No files matched your search
@@ -196,6 +196,16 @@ budget (raised 2026-08-25). Env `scripts/frank-default-large.env`, serve
|
||||
`scripts/serve-frank-large.sh`, runbook
|
||||
[docs/LLM/kingjones-q8agent-rocmfpx.md](docs/LLM/kingjones-q8agent-rocmfpx.md).
|
||||
|
||||
**Serving status (2026-08-25):** the model loads but crashes in
|
||||
`ggml_compute_forward_set_rows` (NULL dispatch in `libggml-cpu.so.0`) before it
|
||||
can serve. Tested on all five available ROCmFPX builds (build-kairic v81,
|
||||
build-promptforge v3, build-rocmi4-w4a4 v257, build-strix-rocmfp4 v209) and the
|
||||
all-CPU path; all crash. Not GPU-specific, not MTP-specific, not fixed by
|
||||
`LLAMA_SET_ROWS=0`. Fix is a rebuild of charlie12345/ROCmFPX with a working
|
||||
`set_rows`. The proven serving context is 131072 at `gl=45` (262144 OOMs on
|
||||
device-local at `gl=999`). Config goals met: 114 GiB TTM/GTT, reasoning on,
|
||||
`preserve_thinking`/`preserve_reasoning` on.
|
||||
|
||||
### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, prior v3, 2026-08-18 to 2026-08-25)
|
||||
|
||||
The v4 runtime serves `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (`eemin` namespace,
|
||||
|
||||
Reference in new issue
Block a user