frank-default-small: use q8_0 (not q8_KV) KV cache on ik_llama

The ik_llama fast q8_KV cache type crashes at model init on this
QWEN3MOE/MTP + multimodal arch at commit b166e269 with a ggml_view size
assert (ggml.c:5427) right after the output buffer is created, looping
the service. Empirically isolated by toggling cache type: q8_0 and f16
load and serve (smoke-tested correct output), q8_KV asserts. Switch to
q8_0 + Hadamard (K/V), which still gives the 8-bit cache + Hadamard
quality win. KB updated with the crash lesson.

docs/LLM/ikllama-ornith-dankserver.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: ik_llama.cpp on Dankserver (Ornith Q6_K)
-description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-default-small on the RTX 3090. q8_KV KV cache with Hadamard, split-mode none (graph requires >=2 GPUs), ngram-mod canonical args, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
+description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-default-small on the RTX 3090. q8_0 KV cache with Hadamard (the fast q8_KV type crashes on this arch at this commit), split-mode none (graph requires >=2 GPUs), ngram-mod canonical args, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
 tags:
   - ikllama
   - llamacpp
@@ -104,14 +104,24 @@ Files: `scripts/serve-frank-small.sh`, `scripts/frank-default-small.env`,
 `scripts/frank-default-small.service` in the repo. The service just calls
 the serve script; the env file holds all tunables.
 
-### KV cache: q8_KV + Hadamard
+### KV cache: q8_0 + Hadamard
 
-- `CACHE_TYPE_K=q8_KV`, `CACHE_TYPE_V=q8_KV` — fast quant type (ik
-  PR 208). Passes `--cache-type-k q8_KV --cache-type-v q8_KV`.
+- `CACHE_TYPE_K=q8_0`, `CACHE_TYPE_V=q8_0` — standard 8-bit KV cache type
+  (passes `--cache-type-k q8_0 --cache-type-v q8_0`).
 - `K_CACHE_HADAMARD=1`, `V_CACHE_HADAMARD=1` — `--k-cache-hadamard` and
   `--v-cache-hadamard`. The Hadamard transform preserves KV quality at
   8-bit (docs recommend it for quantized KV below Q6_0). Implemented for
   both the CUDA (`ggml-cuda/hadamard.cu`) and CPU (IQK) paths.
+- Flash-attention is **on by default** (no `-fa` flag needed).
+
+> ⚠️ **Do not use `q8_KV` (fast type) here.** On this commit (`b166e269`)
+> the fast `q8_KV` cache type crashes at model init on this QWEN3MOE/MTP +
+> multimodal arch with a `ggml view` size assertion
+> (`ggml.c:5427: GGML_ASSERT(view_src == NULL || ... data_size + view_offs`)
+> right after the output buffer is created. It loops the service. Plain
+> `q8_0` (and `f16`) work and were smoke-tested with correct output; this
+> config uses `q8_0`. This was isolated empirically by toggling cache type
+> while keeping the override/hadamard flags.
 
 These keep the 163840-slot / 491520-pool context while roughly halving KV
 cache RAM vs f16.

docs/LLM/index.md

@@ -20,7 +20,7 @@ graph LR
         B2[ik_llama.cpp build b166e269
 frank-default-small, port 7770]
         C[Ornith-1.5-35B-A3B-CRACK Q6_K
-ngram-mod, q8_KV + Hadamard cache, 491520 ctx]
+ngram-mod, q8_0 + Hadamard cache, 491520 ctx]
         A --> B2 --> C
     end
 
@@ -57,7 +57,7 @@ MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
 - [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
   Dankserver (frank-default-small, active 2026-08-24)
 - [[ikllama-ornith-dankserver]] — the ik_llama.cpp runtime for
-  frank-default-small (2026-08-26): q8_KV + Hadamard KV cache,
+  frank-default-small (2026-08-26): q8_0 + Hadamard KV cache,
   split-mode none (single-GPU), mandatory FA_ALL_QUANTS build flags
 - [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
   model on Dankclaw (restored 2026-09-01 after the Q8_0 AGENT experiment)

docs/index.md

@@ -110,7 +110,7 @@ The LLM inference fleet on the two hosts. Full index:
 - [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
 - [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
 - [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)
-- [[ikllama-ornith-dankserver]] — the ik_llama.cpp runtime for frank-default-small (q8_KV + Hadamard KV cache, split-mode none; 2026-08-26)
+- [[ikllama-ornith-dankserver]] — the ik_llama.cpp runtime for frank-default-small (q8_0 + Hadamard KV cache, split-mode none; 2026-08-26)
 - [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
 - [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
 - [[Dankserver System Specs]] — the RTX 3090 box hardware

scripts/frank-default-small.env

@@ -5,12 +5,12 @@
 # 2026-08-26 from commit b166e269) on RTX 3090 24 GB; Ryzen 7 7700X; 62 GB
 # RAM. Previous runtime was upstream llama.cpp b10121. MoE expert weights
 # pinned to CPU with an explicit --override-tensor regex (not --n-cpu-moe).
-# KV cache quantized to q8_KV with Hadamard transforms on both K and V.
-# --split-mode graph enabled (single GPU; retained per operator preference).
+# KV cache quantized to q8_0 with Hadamard transforms on both K and V.
+# Split mode none (single GPU; graph would require 2+ devices).
 #
 # Build flags (IK_FA note, see docs/LLM/ikllama-ornith-dankserver.md) must
 # ALWAYS include: -DGGML_CUDA_FA_ALL_QUANTS=ON and -DGGML_IQK_FA_ALL_QUANTS=ON
-# so the q8_KV + Hadamard KV cache combos compile into the CUDA/CPU FA
+# so the quantized + Hadamard KV cache combos compile into the CUDA/CPU FA
 # kernels. Replacing the HauhauCS Qwen3.6-35B-A3B model on 2026-08-24.
 # Switched from CRACK Q8_0 to Q6_K on 2026-08-25.
 #
@@ -67,12 +67,14 @@ OVERRIDE_BUFT=CPU
 # OVERRIDE_EXPERT below.
 N_GPU_LAYERS=999
 
-# KV cache on GPU, both K and V at q8_KV (fast quant type on ik_llama, PR 208)
-# plus Hadamard transforms on both caches to preserve quality at 8-bit.
-# For quantized KV below Q6_0 the docs recommend --k-cache-hadamard; we apply
-# --v-cache-hadamard to V as well.
-CACHE_TYPE_K=q8_KV
-CACHE_TYPE_V=q8_KV
+# KV cache on GPU, both K and V at q8_0 plus Hadamard transforms on both
+# caches to preserve quality at 8-bit. NOTE: ik_llama's fast q8_KV type
+# crashes at model init on this MTP/multimodal arch with a ggml_view size
+# assert (ggml.c:5427) at commit b166e269; plain q8_0 works. Docs recommend
+# --k-cache-hadamard for quantized KV below Q6_0; we apply --v-cache-hadamard
+# to V as well. Flash-attention is on by default.
+CACHE_TYPE_K=q8_0
+CACHE_TYPE_V=q8_0
 K_CACHE_HADAMARD=1
 V_CACHE_HADAMARD=1
 

scripts/serve-frank-small.sh

@@ -7,11 +7,14 @@
 # 35B-total / 3B-active MoE-dense hybrid with an MTP head bundled in the GGUF;
 # this service uses ngram-mod instead of MTP speculation.
 #
-# KV cache: q8_KV with Hadamard transforms on K and V (requires a build with
+# KV cache: q8_0 with Hadamard transforms on K and V (requires a build with
 # GGML_CUDA_FA_ALL_QUANTS=ON + GGML_IQK_FA_ALL_QUANTS=ON; see the env file).
+# The ik_llama fast q8_KV type crashes at init on this arch (ggml.c:5427);
+# q8_0 is used instead.
 #
 # Split mode is none: Dankserver has a single RTX 3090 and ik_llama rejects
 # split-mode graph with fewer than 2 devices (hard load error).
+
 #
 # Expert offload: the expert (FFN exps) tensors selected by the layer-bounded
 # --override-tensor pattern are pinned to CPU. Selected full-attention and