frank-default-small: use q8_0 (not q8_KV) KV cache on ik_llama

The ik_llama fast q8_KV cache type crashes at model init on this
QWEN3MOE/MTP + multimodal arch at commit b166e269 with a ggml_view size
assert (ggml.c:5427) right after the output buffer is created, looping
the service. Empirically isolated by toggling cache type: q8_0 and f16
load and serve (smoke-tested correct output), q8_KV asserts. Switch to
q8_0 + Hadamard (K/V), which still gives the 8-bit cache + Hadamard
quality win. KB updated with the crash lesson.
This commit is contained in:
Frank committed 2026-08-26 03:23:18 -07:00
1 parent e2085f1f7d
commit 0e5c37f2f8
5 files changed
+32 -17

No files matched your search

+2 -2
View File
@@ -20,7 +20,7 @@ graph LR
B2[ik_llama.cpp build b166e269
frank-default-small, port 7770]
C[Ornith-1.5-35B-A3B-CRACK Q6_K
ngram-mod, q8_KV + Hadamard cache, 491520 ctx]
ngram-mod, q8_0 + Hadamard cache, 491520 ctx]
A --> B2 --> C
end
@@ -57,7 +57,7 @@ MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
Dankserver (frank-default-small, active 2026-08-24)
- [[ikllama-ornith-dankserver]] — the ik_llama.cpp runtime for
frank-default-small (2026-08-26): q8_KV + Hadamard KV cache,
frank-default-small (2026-08-26): q8_0 + Hadamard KV cache,
split-mode none (single-GPU), mandatory FA_ALL_QUANTS build flags
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
model on Dankclaw (restored 2026-09-01 after the Q8_0 AGENT experiment)