Swap LLM fleet to Ornith CRACK (small) and kingjones ROCmFP4 STRIX MTP (large); apply Qwen-Sharp chat template

- frank-default-small (Dankserver): Ornith-1.5-35B-A3B-CRACK Q8_0, bundled
  native draft-mtp (n_max 4), MoE expert FFN tensors pinned to CPU via an
  explicit --override-tensor \.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU
  (replaces --n-cpu-moe), Qwen-Sharp chat template.
- frank-default-large (Dankclaw): kingjones Qwen3.8-27B-Uncensored-Q6_0
  ROCmFP4 AGENT dense GGUF, standalone mtp-Qwen3.8-27B-Q4_0 draft head
  (--model-draft, draft-mtp n_max 4), Qwen-Sharp chat template. Drop the
  retired PromptForge .pfs sidecar lanes.
- Track scripts/chat-template.jinja (v22.4.0 Sharp) and apply via
  --chat-template-file on both services.
- Docs: new kingjones-strix-mtp and dealign-ornith-crack runbook pages;
  update index, deployment, and both system specs; models.json names.

Smoke-tested both models on the real hardware on scratch ports before
commit: model loads, MTP draft attaches, Sharp template applies, chat
completion returns.
This commit is contained in:
Frank committed 2026-08-24 21:35:11 -07:00
1 parent 189934af15
commit 4462aa992c
14 files changed
+827 -156

No files matched your search

+23 -20
View File
@@ -17,8 +17,8 @@ repository to a running service.
| Target | Host | Runtime | Service | Port |
|--------|------|---------|---------|------|
| Dankserver | dank@dankserver | upstream llama.cpp (Qwen3.6-35B-A3B Q6_K_P + ngram-mod, MoE on CPU) | frank-default-small | 7770 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3, native MTP + Output-K8 proxy + PromptForge DualView, 262K ctx / 1 slot) | frank-default-large | 7770 |
| Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q8_0, native MTP draft, MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (Qwen3.8-27B-Uncensored-ROCmFP4 Q6 AGENT, standalone MTP Q4_0 head, 262K ctx / 1 slot) | frank-default-large | 7770 |
## Workflow
@@ -122,32 +122,35 @@ curl -sf http://127.0.0.1:7770/health
### Dankserver — frank-default-small
The active service on Dankserver is upstream llama.cpp (b10121) serving
`Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q6_K_P.gguf` under the
alias `frank-default-small` on port 7770. It runs the stock
`/home/dank/llama.cpp/build/bin/llama-server` binary with the 38 MoE
experts on CPU (`--n-cpu-moe 38`) and 7 CPU threads, ngram-mod
speculative decoding (n_min 48 / n_max 64 / n_match 24), f16 KV for both
K and V, context 786432 (3 slots of ~262K each), batch 4096 / ubatch 2048,
`--no-mmap`. The unit file is `scripts/frank-default-small.service`. It
launches through `scripts/serve-frank-small.sh`, which reads tunables from
`scripts/frank-default-small.env`.
`Ornith-1.5-35B-A3B-CRACK-Q8_0.gguf` under the alias `frank-default-small`
on port 7770. It runs the stock `/home/dank/llama.cpp/build/bin/llama-server`
binary with the MoE expert weights pinned to CPU by an explicit
`--override-tensor` regex (the expert `\.ffn_(up|down|gate|gate_up)_(ch|)exps`
pattern, buffer type `CPU`) and 7 CPU threads, native `draft-mtp`
speculative decoding (bundled MoE MTP head, n_max 4), f16 KV, context
786432 (3 slots of ~262K each), batch 4096 / ubatch 2048, `--no-mmap`. The
unit is `scripts/frank-default-small.service`. It launches through
`scripts/serve-frank-small.sh`, which reads tunables from
`scripts/frank-default-small.env`. See [[dealign-ornith-crack]].
### Dankclaw — frank-default-large
The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` (v3 five-file payload: GGUF
+ FFN/GDN .pfs + Output-K8 proxy + BF16 mmproj) under the alias
`frank-default-large` on port 7770 (moved from 52625 on 2026-08-22). It runs
`kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` (Q6_0 AGENT, dense)
under the alias `frank-default-large` on port 7770. It runs
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context
(1 slot. Concurrent requests queue. The server never preempts a running
generation), batch 4096 / ubatch 4096, 16 context checkpoints, 8 GiB
prompt cache, and reasoning in deepseek format. The unit file is
7.15 SDK in the retired ling3-ciru venv) with a standalone MTP Q4_0 draft
head (`--model-draft` and `--spec-type draft-mtp`), f16 KV, full 262144
context (1 slot. Concurrent requests queue; the server never preempts a
running generation), batch 4096 / ubatch 4096, 16 context checkpoints,
8 GiB prompt cache, and reasoning in deepseek format. The unit file is
`scripts/frank-default-large.service`. It launches through
`scripts/serve-frank-large.sh`, which reads tunables from
`scripts/frank-default-large.env`. See [[ciru-activefpx-promptforge]].
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
See [[kingjones-strix-mtp]].
> The kingjones Q6 AGENT model has served the slot since 2026-08-24; the
> v3 PromptForge payload is retired on this slot.
> Restored 2026-08-21. The intermediate DFlash2 Vulkan deployment was
> retired the same day. See [[dflash2-large-vulkan]] for the archived runbook.