Swap LLM fleet to Ornith CRACK (small) and kingjones ROCmFP4 STRIX MTP (large); apply Qwen-Sharp chat template
- frank-default-small (Dankserver): Ornith-1.5-35B-A3B-CRACK Q8_0, bundled native draft-mtp (n_max 4), MoE expert FFN tensors pinned to CPU via an explicit --override-tensor \.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU (replaces --n-cpu-moe), Qwen-Sharp chat template. - frank-default-large (Dankclaw): kingjones Qwen3.8-27B-Uncensored-Q6_0 ROCmFP4 AGENT dense GGUF, standalone mtp-Qwen3.8-27B-Q4_0 draft head (--model-draft, draft-mtp n_max 4), Qwen-Sharp chat template. Drop the retired PromptForge .pfs sidecar lanes. - Track scripts/chat-template.jinja (v22.4.0 Sharp) and apply via --chat-template-file on both services. - Docs: new kingjones-strix-mtp and dealign-ornith-crack runbook pages; update index, deployment, and both system specs; models.json names. Smoke-tested both models on the real hardware on scratch ports before commit: model loads, MTP draft attaches, Sharp template applies, chat completion returns.
This commit is contained in:
1 parent
189934af15
commit
4462aa992c
14 files changed
+827
-156
No files matched your search
@@ -60,7 +60,7 @@ Dankserver is an NVIDIA GPU server for vLLM inference. The host runs Ubuntu 26.0
|
||||
|
||||
### Active Services
|
||||
|
||||
The active service on Dankserver is `frank-default-small.service`: the llama.cpp Qwen3.6-35B-A3B (Q6_K_P + ngram-mod) on port 7770. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
|
||||
The active service on Dankserver is `frank-default-small.service`: the llama.cpp Ornith-1.5-35B-A3B CRACK (Q8_0) on port 7770, with MoE expert weights held on CPU via `--override-tensor`. The earlier PP worker (`frank-pp-worker.service`) was retired with the 2026-08-17 redo. See [[deployment]].
|
||||
|
||||
## Hardware Relevance
|
||||
|
||||
|
||||
Reference in new issue
Block a user