frank-default-small: run on ik_llama.cpp with q8_KV + Hadamard cache
Switch frank-default-small on Dankserver from upstream llama.cpp (b10121) to ikawrakow/ik_llama.cpp (b166e269, built with the staged pip cu13 toolkit). The model (Ornith-1.5-35B-A3B CRACK Q6_K) is unchanged and the old binary is kept for rollback. Runtime changes: - KV cache quantized to q8_KV with --k-cache-hadamard/--v-cache-hadamard - --split-mode graph (single-GPU no-op, retained per operator pref) - migrate ngram speculation to canonical --spec-type ngram-mod:n_min=48,n_max=64,ngram_size_n=24 (legacy --spec-ngram-mod-* are rejected in ik main) - N_GPU_LAYERS 999 (ik -ngl rejects the literal "all"); experts stay pinned to CPU via --override-tensor Build: GGML_CUDA_FA_ALL_QUANTS=ON + GGML_IQK_FA_ALL_QUANTS=ON mandatory for the quantized KV cache; recipe + reasoning baked into docs/LLM/ikllama-ornith.md. Tests and both indices updated.
This commit is contained in:
1 parent
c8d7ed65c6
commit
b8de25d70b
6 files changed
+295
-45
No files matched your search
+2
-1
@@ -110,9 +110,10 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)
|
||||
- [[ikllama-ornith-dankserver]] — the ik_llama.cpp runtime for frank-default-small (q8_KV + Hadamard KV cache, split-mode graph; 2026-08-26)
|
||||
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment (2026-08-22)
|
||||
- [[dflash2-large-vulkan]] — the retired DFlash2 Vulkan deployment (2026-08-21)
|
||||
- [[Dankserver System Specs]] — the upstream llama.cpp box hardware
|
||||
- [[Dankserver System Specs]] — the RTX 3090 box hardware
|
||||
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box hardware
|
||||
- [[Dankclaw Improvements]] — host-level changes on the CIRU box
|
||||
|
||||
|
||||
Reference in new issue
Block a user