Swap LLM fleet to Ornith CRACK (small) and kingjones ROCmFP4 STRIX MTP (large); apply Qwen-Sharp chat template

- frank-default-small (Dankserver): Ornith-1.5-35B-A3B-CRACK Q8_0, bundled
  native draft-mtp (n_max 4), MoE expert FFN tensors pinned to CPU via an
  explicit --override-tensor \.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU
  (replaces --n-cpu-moe), Qwen-Sharp chat template.
- frank-default-large (Dankclaw): kingjones Qwen3.8-27B-Uncensored-Q6_0
  ROCmFP4 AGENT dense GGUF, standalone mtp-Qwen3.8-27B-Q4_0 draft head
  (--model-draft, draft-mtp n_max 4), Qwen-Sharp chat template. Drop the
  retired PromptForge .pfs sidecar lanes.
- Track scripts/chat-template.jinja (v22.4.0 Sharp) and apply via
  --chat-template-file on both services.
- Docs: new kingjones-strix-mtp and dealign-ornith-crack runbook pages;
  update index, deployment, and both system specs; models.json names.

Smoke-tested both models on the real hardware on scratch ports before
commit: model loads, MTP draft attaches, Sharp template applies, chat
completion returns.
This commit is contained in:
Frank committed 2026-08-24 21:35:11 -07:00
1 parent 189934af15
commit 4462aa992c
14 files changed
+827 -156

No files matched your search

+1 -1
View File
@@ -60,7 +60,7 @@ Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.0
### Active Services
The active service on Dankserver is `frank-default-small.service`: the llama.cpp Qwen3.6-35B-A3B (Q6_K_P + ngram-mod) on port 7770. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
The active service on Dankserver is `frank-default-small.service`: the llama.cpp Ornith-1.5-35B-A3B CRACK (Q8_0) on port 7770, with MoE expert weights held on CPU via `--override-tensor`. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
## Hardware Relevance