Swap LLM fleet to Ornith CRACK (small) and kingjones ROCmFP4 STRIX MTP (large); apply Qwen-Sharp chat template
- frank-default-small (Dankserver): Ornith-1.5-35B-A3B-CRACK Q8_0, bundled native draft-mtp (n_max 4), MoE expert FFN tensors pinned to CPU via an explicit --override-tensor \.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU (replaces --n-cpu-moe), Qwen-Sharp chat template. - frank-default-large (Dankclaw): kingjones Qwen3.8-27B-Uncensored-Q6_0 ROCmFP4 AGENT dense GGUF, standalone mtp-Qwen3.8-27B-Q4_0 draft head (--model-draft, draft-mtp n_max 4), Qwen-Sharp chat template. Drop the retired PromptForge .pfs sidecar lanes. - Track scripts/chat-template.jinja (v22.4.0 Sharp) and apply via --chat-template-file on both services. - Docs: new kingjones-strix-mtp and dealign-ornith-crack runbook pages; update index, deployment, and both system specs; models.json names. Smoke-tested both models on the real hardware on scratch ports before commit: model loads, MTP draft attaches, Sharp template applies, chat completion returns.
This commit is contained in:
1 parent
189934af15
commit
4462aa992c
14 files changed
+827
-156
No files matched your search
+23
-20
@@ -17,8 +17,8 @@ repository to a running service.
|
||||
|
||||
| Target | Host | Runtime | Service | Port |
|
||||
|--------|------|---------|---------|------|
|
||||
| Dankserver | dank@dankserver | upstream llama.cpp (Qwen3.6-35B-A3B Q6_K_P + ngram-mod, MoE on CPU) | frank-default-small | 7770 |
|
||||
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3, native MTP + Output-K8 proxy + PromptForge DualView, 262K ctx / 1 slot) | frank-default-large | 7770 |
|
||||
| Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q8_0, native MTP draft, MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
|
||||
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (Qwen3.8-27B-Uncensored-ROCmFP4 Q6 AGENT, standalone MTP Q4_0 head, 262K ctx / 1 slot) | frank-default-large | 7770 |
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -122,32 +122,35 @@ curl -sf http://127.0.0.1:7770/health
|
||||
### Dankserver — frank-default-small
|
||||
|
||||
The active service on Dankserver is upstream llama.cpp (b10121) serving
|
||||
`Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q6_K_P.gguf` under the
|
||||
alias `frank-default-small` on port 7770. It runs the stock
|
||||
`/home/dank/llama.cpp/build/bin/llama-server` binary with the 38 MoE
|
||||
experts on CPU (`--n-cpu-moe 38`) and 7 CPU threads, ngram-mod
|
||||
speculative decoding (n_min 48 / n_max 64 / n_match 24), f16 KV for both
|
||||
K and V, context 786432 (3 slots of ~262K each), batch 4096 / ubatch 2048,
|
||||
`--no-mmap`. The unit file is `scripts/frank-default-small.service`. It
|
||||
launches through `scripts/serve-frank-small.sh`, which reads tunables from
|
||||
`scripts/frank-default-small.env`.
|
||||
`Ornith-1.5-35B-A3B-CRACK-Q8_0.gguf` under the alias `frank-default-small`
|
||||
on port 7770. It runs the stock `/home/dank/llama.cpp/build/bin/llama-server`
|
||||
binary with the MoE expert weights pinned to CPU by an explicit
|
||||
`--override-tensor` regex (the expert `\.ffn_(up|down|gate|gate_up)_(ch|)exps`
|
||||
pattern, buffer type `CPU`) and 7 CPU threads, native `draft-mtp`
|
||||
speculative decoding (bundled MoE MTP head, n_max 4), f16 KV, context
|
||||
786432 (3 slots of ~262K each), batch 4096 / ubatch 2048, `--no-mmap`. The
|
||||
unit is `scripts/frank-default-small.service`. It launches through
|
||||
`scripts/serve-frank-small.sh`, which reads tunables from
|
||||
`scripts/frank-default-small.env`. See [[dealign-ornith-crack]].
|
||||
|
||||
### Dankclaw — frank-default-large
|
||||
|
||||
The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
|
||||
`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` (v3 five-file payload: GGUF
|
||||
+ FFN/GDN .pfs + Output-K8 proxy + BF16 mmproj) under the alias
|
||||
`frank-default-large` on port 7770 (moved from 52625 on 2026-08-22). It runs
|
||||
`kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` (Q6_0 AGENT, dense)
|
||||
under the alias `frank-default-large` on port 7770. It runs
|
||||
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
|
||||
7.15 SDK in the retired ling3-ciru venv) with native depth-4 MTP
|
||||
(`--spec-type draft-mtp`), f16 KV for both K and V, full 262144 context
|
||||
(1 slot. Concurrent requests queue. The server never preempts a running
|
||||
generation), batch 4096 / ubatch 4096, 16 context checkpoints, 8 GiB
|
||||
prompt cache, and reasoning in deepseek format. The unit file is
|
||||
7.15 SDK in the retired ling3-ciru venv) with a standalone MTP Q4_0 draft
|
||||
head (`--model-draft` and `--spec-type draft-mtp`), f16 KV, full 262144
|
||||
context (1 slot. Concurrent requests queue; the server never preempts a
|
||||
running generation), batch 4096 / ubatch 4096, 16 context checkpoints,
|
||||
8 GiB prompt cache, and reasoning in deepseek format. The unit file is
|
||||
`scripts/frank-default-large.service`. It launches through
|
||||
`scripts/serve-frank-large.sh`, which reads tunables from
|
||||
`scripts/frank-default-large.env`. See [[ciru-activefpx-promptforge]].
|
||||
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
|
||||
See [[kingjones-strix-mtp]].
|
||||
|
||||
> The kingjones Q6 AGENT model has served the slot since 2026-08-24; the
|
||||
> v3 PromptForge payload is retired on this slot.
|
||||
> Restored 2026-08-21. The intermediate DFlash2 Vulkan deployment was
|
||||
> retired the same day. See [[dflash2-large-vulkan]] for the archived runbook.
|
||||
|
||||
|
||||
Reference in new issue
Block a user