frank-default-small: Q8_0 model, 160k single slot, q6_0 KV + Hadamard

Switch to CRACK Q8_0 model (37.8 GB, mmproj f16). Servt as a single
160000-ctx slot (PARALLEL=1); speculative decoding stays disabled. KV
cache is q6_0 + Hadamard. iq5_k is not a valid KV-cache type in ik_llama
(cache whitelist excludes iq5_*); q8_KV reproduces the ggml.c:5427 view
assert even at PARALLEL=1/no-spec, so q6_0 is the verified working 6-bit
choice. Env, serve script, tests, and KB updated. Deployment was left
down after the isolation test; this commit redeploys it with the
verified Q8_0/160k/q6_0 config.

docs/LLM/ikllama-ornith-dankserver.md

@@ -1,7 +1,7 @@
 ---
 type: reference
-title: ik_llama.cpp on Dankserver (Ornith Q6_K)
-description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-default-small on the RTX 3090. q8_0 KV cache with Hadamard (the fast q8_KV type crashes on this arch at this commit), split-mode none (graph requires >=2 GPUs), speculative decoding disabled (3 parallel slots), the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
+title: ik_llama.cpp on Dankserver (Ornith Q8_0)
+description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-default-small on the RTX 3090. Q8_0 model, q6_0 KV cache with Hadamard (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 160k single slot, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
 tags:
   - ikllama
   - llamacpp
@@ -21,11 +21,11 @@ timestamp: 2026-08-26
 > RTX 3090. This page documents the build, the mandatory FA build flags, and
 > the MoE / CPU-offload runtime arguments.
 
-The model is unchanged: `dealignai/Ornith-1.5-35B-A3B-UNCENSORED` (CRACK
-Q6_K), 35B-total / 3B-active MoE-dense hybrid with an MTP head in the GGUF.
-Speculative decoding is **disabled** (ik_llama limits spec to a single slot;
-this service keeps 3 parallel slots). The low-parameter MoE does not need
-it. See [[dealign-ornith-crack]] for the model runbook.
+The model is `dealignai/Ornith-1.5-35B-A3B-UNCENSORED` (CRACK **Q8_0**), 35B-total /
+3B-active MoE-dense hybrid with an MTP head in the GGUF (mmproj f16 vision
+encoder). Served as a **single 160000-context slot** (`PARALLEL=1`).
+Speculative decoding is **disabled**. See [[dealign-ornith-crack]] for the
+model runbook.
 
 ## Why ik_llama.cpp
 
@@ -105,27 +105,27 @@ Files: `scripts/serve-frank-small.sh`, `scripts/frank-default-small.env`,
 `scripts/frank-default-small.service` in the repo. The service just calls
 the serve script; the env file holds all tunables.
 
-### KV cache: q8_0 + Hadamard
+### KV cache: q8_0 → q6_0 + Hadamard
 
-- `CACHE_TYPE_K=q8_0`, `CACHE_TYPE_V=q8_0` — standard 8-bit KV cache type
-  (passes `--cache-type-k q8_0 --cache-type-v q8_0`).
+- `CACHE_TYPE_K=q6_0`, `CACHE_TYPE_V=q6_0` — 6-bit k-quant KV cache
+  (passes `--cache-type-k q6_0 --cache-type-v q6_0`).
 - `K_CACHE_HADAMARD=1`, `V_CACHE_HADAMARD=1` — `--k-cache-hadamard` and
   `--v-cache-hadamard`. The Hadamard transform preserves KV quality at
-  8-bit (docs recommend it for quantized KV below Q6_0). Implemented for
-  both the CUDA (`ggml-cuda/hadamard.cu`) and CPU (IQK) paths.
-- Flash-attention is **on by default** (no `-fa` flag needed).
+  6-bit. Implemented for both CUDA (`ggml-cuda/hadamard.cu`) and CPU (IQK)
+  paths. Flash-attention is **on by default**.
 
-> ⚠️ **Do not use `q8_KV` (fast type) here.** On this commit (`b166e269`)
-> the fast `q8_KV` cache type crashes at model init on this QWEN3MOE/MTP +
-> multimodal arch with a `ggml view` size assertion
-> (`ggml.c:5427: GGML_ASSERT(view_src == NULL || ... data_size + view_offs`)
-> right after the output buffer is created. It loops the service. Plain
-> `q8_0` (and `f16`) work and were smoke-tested with correct output; this
-> config uses `q8_0`. This was isolated empirically by toggling cache type
-> while keeping the override/hadamard flags.
+> **KV cache type history on this build (`b166e269`):**
+> - `iq5_k` is **not** a KV-cache type — the cache whitelist is
+>   `f32/f16/bf16/q8_0/q4_0/q4_1/iq4_nl/q5_0/q5_1/q6_0/q8_KV`; `iq5_*` only
+>   exist as weight quants. `--cache-type-k iq5_k` throws `Invalid cache
+>   type`.
+> - `q8_KV` **crashes** with a `ggml view` size assert
+>   (`ggml.c:5427: data_size + view_offs > ggml_nbytes(view_src)`) on this
+>   QWEN3MOE/MTP + multimodal arch — independent of PARALLEL/speculation.
+> - **`q6_0` + Hadamard works** and is what this deployment uses.
 
-These keep the 163840-slot / 491520-pool context while roughly halving KV
-cache RAM vs f16.
+At 160k single slot the q6_0 KV self is ~1270 MiB (K 634 + V 634), leaving
+~2.2 GiB headroom on the 24 GiB card after the GPU-side experts.
 
 ### Split mode: none (single GPU)