frank-default-small: use q8_0 (not q8_KV) KV cache on ik_llama

The ik_llama fast q8_KV cache type crashes at model init on this
QWEN3MOE/MTP + multimodal arch at commit b166e269 with a ggml_view size
assert (ggml.c:5427) right after the output buffer is created, looping
the service. Empirically isolated by toggling cache type: q8_0 and f16
load and serve (smoke-tested correct output), q8_KV asserts. Switch to
q8_0 + Hadamard (K/V), which still gives the 8-bit cache + Hadamard
quality win. KB updated with the crash lesson.
This commit is contained in:
Frank committed 2026-08-26 03:23:18 -07:00
1 parent e2085f1f7d
commit 0e5c37f2f8
5 files changed
+32 -17

No files matched your search

+1 -1
View File
@@ -110,7 +110,7 @@ The LLM inference fleet on the two hosts. Full index:
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)
- [[ikllama-ornith-dankserver]] — the ik_llama.cpp runtime for frank-default-small (q8_KV + Hadamard KV cache, split-mode none; 2026-08-26)
- [[ikllama-ornith-dankserver]] — the ik_llama.cpp runtime for frank-default-small (q8_0 + Hadamard KV cache, split-mode none; 2026-08-26)
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
- [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
- [[Dankserver System Specs]] — the RTX 3090 box hardware