frank-default-small: run on ik_llama.cpp with q8_KV + Hadamard cache

Switch frank-default-small on Dankserver from upstream llama.cpp (b10121)
to ikawrakow/ik_llama.cpp (b166e269, built with the staged pip cu13
toolkit). The model (Ornith-1.5-35B-A3B CRACK Q6_K) is unchanged and the
old binary is kept for rollback.

Runtime changes:
- KV cache quantized to q8_KV with --k-cache-hadamard/--v-cache-hadamard
- --split-mode graph (single-GPU no-op, retained per operator pref)
- migrate ngram speculation to canonical
  --spec-type ngram-mod:n_min=48,n_max=64,ngram_size_n=24
  (legacy --spec-ngram-mod-* are rejected in ik main)
- N_GPU_LAYERS 999 (ik -ngl rejects the literal "all"); experts stay
  pinned to CPU via --override-tensor

Build: GGML_CUDA_FA_ALL_QUANTS=ON + GGML_IQK_FA_ALL_QUANTS=ON mandatory
for the quantized KV cache; recipe + reasoning baked into
docs/LLM/ikllama-ornith.md. Tests and both indices updated.
This commit is contained in:
Frank committed 2026-08-26 02:59:34 -07:00
1 parent c8d7ed65c6
commit b8de25d70b
6 files changed
+295 -45

No files matched your search

+2 -1
View File
@@ -110,9 +110,10 @@ The LLM inference fleet on the two hosts. Full index:
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)
- [[ikllama-ornith-dankserver]] — the ik_llama.cpp runtime for frank-default-small (q8_KV + Hadamard KV cache, split-mode graph; 2026-08-26)
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
- [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
- [[Dankserver System Specs]] — the upstream llama.cpp box hardware
- [[Dankserver System Specs]] — the RTX 3090 box hardware
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box hardware
- [[Dankclaw Improvements]] — host-level changes on the CIRU box