frank-default-small: PARALLEL=3 160k, KV K q8_0 / V q5_0 + Hadamard

Reconfigure to 3 parallel 160000-ctx slots (was 1 slot). KV cache is
now asymmetric: K at q8_0, V at q5_0, both with Hadamard. The 3 slots
triple the KV, so expert blocks 27-34 are pinned back to CPU (freed
~8.7 GiB) leaving output-adjacent 35-40 on CUDA. Q8_0 model,
split-mode none, spec disabled (ik limits spec to 1 slot). AGENTS.md:
drop the scratch-port rule (test/bench processes cannot share the
single GPU with the running service anyway). Env, serve script, bats,
docs updated.

docs/LLM/ikllama-ornith-dankserver.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: ik_llama.cpp on Dankserver (Ornith Q8_0)
-description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-default-small on the RTX 3090. Q8_0 model, q6_0 KV cache with Hadamard (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 160k single slot, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
+description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-default-small on the RTX 3090. Q8_0 model, KV K q8_0 / V q5_0 with Hadamard (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 160k ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
 tags:
   - ikllama
   - llamacpp
@@ -23,7 +23,7 @@ timestamp: 2026-08-26
 
 The model is `dealignai/Ornith-1.5-35B-A3B-UNCENSORED` (CRACK **Q8_0**), 35B-total /
 3B-active MoE-dense hybrid with an MTP head in the GGUF (mmproj f16 vision
-encoder). Served as a **single 160000-context slot** (`PARALLEL=1`).
+encoder). Served as **three parallel 160000-context slots** (`PARALLEL=3`).
 Speculative decoding is **disabled**. See [[dealign-ornith-crack]] for the
 model runbook.
 
@@ -105,14 +105,16 @@ Files: `scripts/serve-frank-small.sh`, `scripts/frank-default-small.env`,
 `scripts/frank-default-small.service` in the repo. The service just calls
 the serve script; the env file holds all tunables.
 
-### KV cache: q8_0 → q6_0 + Hadamard
+### KV cache: K q8_0 / V q5_0 + Hadamard
 
-- `CACHE_TYPE_K=q6_0`, `CACHE_TYPE_V=q6_0` — 6-bit k-quant KV cache
-  (passes `--cache-type-k q6_0 --cache-type-v q6_0`).
+- `CACHE_TYPE_K=q8_0`, `CACHE_TYPE_V=q5_0` — asymmetric KV precision: the
+  K cache at 8-bit, the V cache at 5-bit (passes `--cache-type-k q8_0
+  --cache-type-v q5_0`). Keeping K higher-precision than V is a common
+  asymmetric budget split.
 - `K_CACHE_HADAMARD=1`, `V_CACHE_HADAMARD=1` — `--k-cache-hadamard` and
   `--v-cache-hadamard`. The Hadamard transform preserves KV quality at
-  6-bit. Implemented for both CUDA (`ggml-cuda/hadamard.cu`) and CPU (IQK)
-  paths. Flash-attention is **on by default**.
+  quantized precision. Implemented for both CUDA (`ggml-cuda/hadamard.cu`)
+  and CPU (IQK) paths. Flash-attention is **on by default**.
 
 > **KV cache type history on this build (`b166e269`):**
 > - `iq5_k` is **not** a KV-cache type — the cache whitelist is
@@ -122,10 +124,12 @@ the serve script; the env file holds all tunables.
 > - `q8_KV` **crashes** with a `ggml view` size assert
 >   (`ggml.c:5427: data_size + view_offs > ggml_nbytes(view_src)`) on this
 >   QWEN3MOE/MTP + multimodal arch — independent of PARALLEL/speculation.
-> - **`q6_0` + Hadamard works** and is what this deployment uses.
+> - `q6_0` (both caches) boots and served, but with `PARALLEL=3` the KV
+>   tripled, so the K cache went back up to `q8_0` and V down to `q5_0` for
+>   the chosen K/V budget split.
 
-At 160k single slot the q6_0 KV self is ~1270 MiB (K 634 + V 634), leaving
-~2.2 GiB headroom on the 24 GiB card after the GPU-side experts.
+With 3 parallel slots at 160k context, the KV cache is ~3x a single slot's.
+The GPU-side expert set is reduced to fit (see the expert section).
 
 ### Split mode: none (single GPU)
 
@@ -148,15 +152,17 @@ At 160k single slot the q6_0 KV self is ~1270 MiB (K 634 + V 634), leaving
 
 The expert-offload pattern from upstream llama.cpp carries over: the
 `--override-tensor` regex pins the sparse expert (`ffn_*_exps`) tensors to
-the CPU buft, leaving every full-attention block plus blocks 35-40 on CUDA.
-On 2026-08-26 we brought expert blocks **30, 32, 33, 34** back to CUDA
-(~2.5 GiB), then **28 and 29** (~1.26 GiB) for the fused-MoE run, to use the
-GPU headroom for lower latency; the rest of that CPU range stays on CPU.
+the CPU buft, leaving the output-adjacent blocks 35-40 on CUDA. On the
+Q8_0 model each ffn expert tensor is **272 MiB** (~4 tensor/block ≈ 1088
+MiB/block). With `PARALLEL=3` the KV cache tripled, so most expert blocks
+are pinned to CPU; only blocks **35-40** (output-adjacent, latency- and
+quality-relevant) stay on CUDA. Full-attention blocks 3,7,11,15,19,23 have
+no exps and are untouched by the regex.
 
-- `OVERRIDE_EXPERT` — the layer-bounded regex listing only the blocks whose
+- `OVERRIDE_EXPERT` — the layer-bounded regex listing the blocks whose
   experts stay on CPU: `(0|1|2|4|5|6|8|9|10|12|13|14|16|17|18|20|21|22|
-  24|25|26)`. Blocks 28,29,30,32,33,34 (and the always-on full-attention +
-  35-40 set) are left off the list so their experts load to CUDA.
+  24|25|26|27|28|29|30|31|32|33|34)`. Blocks 35-40 (and the always-on
+  full-attention set) are left off the list so their experts load to CUDA.
 - `OVERRIDE_BUFT=CPU`.
 - `N_GPU_LAYERS=999` — ik_llama `-ngl` / `--gpu-layers` only accepts an
   integer (the literal `all` is rejected). 999 means "load as many tensor
@@ -169,10 +175,9 @@ GPU headroom for lower latency; the rest of that CPU range stays on CPU.
 > boot log for the running service.
 
 To move more experts back to the GPU later, drop more block numbers from
-`OVERRIDE_EXPERT` (each block is ~630 MiB; verify `nvidia-smi` headroom
-first and keep ~2-3 GiB margin for KV/compute). After 28,29 were added the
-free headroom was ~1.6 GiB, which is tighter than ideal — add more only
-with a headroom check.
+`OVERRIDE_EXPERT` (each block is ~1088 MiB on Q8_0; verify `nvidia-smi`
+headroom first and keep ~2-3 GiB margin for KV/compute). With the 3-slot KV
+cache there is little headroom, so the default keeps only 35-40 on GPU.
 
 ### Qwen-Sharp chat template via --jinja