frank-default-small: use q8_0 (not q8_KV) KV cache on ik_llama
The ik_llama fast q8_KV cache type crashes at model init on this QWEN3MOE/MTP + multimodal arch at commit b166e269 with a ggml_view size assert (ggml.c:5427) right after the output buffer is created, looping the service. Empirically isolated by toggling cache type: q8_0 and f16 load and serve (smoke-tested correct output), q8_KV asserts. Switch to q8_0 + Hadamard (K/V), which still gives the 8-bit cache + Hadamard quality win. KB updated with the crash lesson.
This commit is contained in:
1 parent
e2085f1f7d
commit
0e5c37f2f8
5 files changed
+32
-17
No files matched your search
+2
-2
@@ -20,7 +20,7 @@ graph LR
|
||||
B2[ik_llama.cpp build b166e269
|
||||
frank-default-small, port 7770]
|
||||
C[Ornith-1.5-35B-A3B-CRACK Q6_K
|
||||
ngram-mod, q8_KV + Hadamard cache, 491520 ctx]
|
||||
ngram-mod, q8_0 + Hadamard cache, 491520 ctx]
|
||||
A --> B2 --> C
|
||||
end
|
||||
|
||||
@@ -57,7 +57,7 @@ MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
|
||||
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
|
||||
Dankserver (frank-default-small, active 2026-08-24)
|
||||
- [[ikllama-ornith-dankserver]] — the ik_llama.cpp runtime for
|
||||
frank-default-small (2026-08-26): q8_KV + Hadamard KV cache,
|
||||
frank-default-small (2026-08-26): q8_0 + Hadamard KV cache,
|
||||
split-mode none (single-GPU), mandatory FA_ALL_QUANTS build flags
|
||||
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
|
||||
model on Dankclaw (restored 2026-09-01 after the Q8_0 AGENT experiment)
|
||||
|
||||
Reference in new issue
Block a user