dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

Restore Q6 AGENT large model

Browse Source
This commit is contained in:
Frank
2026-08-25 13:36:07 -07:00
parent 205b11aae4
commit 86d5f38142
8 changed files with 51 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/LLM/deployment.md
+10 -10
View File
@@ -18,7 +18,7 @@ repository to a running service.
| Target | Host | Runtime | Service | Port |
|--------|------|---------|---------|------|
| Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q6_K, native MTP draft, selected MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (caf-eemin Qwen3.8-27B ROCmI4 Q4_0, embedded MTP, ROCmI4 W4A4, 262K ctx / 1 slot) | frank-default-large | 7770 |
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (kingjones Qwen3.8 Uncensored Q6_0 ROCmFPX AGENT, standalone MTP, 262K ctx / 1 slot) | frank-default-large | 7770 |
## Workflow
@@ -138,21 +138,21 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
### Dankclaw — frank-default-large
The active service on Dankclaw uses the ROCmFPX llama.cpp runtime to serve
[kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF)
[kingjones777/Qwen3.8-Uncensored-ROCmFP4-STRIX-MTP-GGUF](https://huggingface.co/kingjones777/Qwen3.8-Uncensored-ROCmFP4-STRIX-MTP-GGUF)
under the alias `frank-default-large` on port 7770. The main model is
`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the BF16 vision projector is loaded alongside
`Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the Q8_0 vision projector is loaded alongside
it. The K-side KV cache remains `f16`; the V-side cache uses `turbo4`. The
service keeps the existing 65536 context / 1-slot profile. The unit file is
service uses the 262144 context / 1-slot profile. The unit file is
`scripts/frank-default-large.service`; it launches through
`scripts/serve-frank-large.sh`, which reads tunables from
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
> The kingjones ROCmFP4 STRIX MTP model served the slot from 2026-08-24; it
> was replaced by the caf-eemin ROCmI4 embedded-MTP model on 2026-08-25. The
> v3 PromptForge payload and the CIRU ROCmFPX runtime are retired on this slot.
> The intermediate DFlash2 Vulkan deployment was retired the same day (2026-08-21).
> See [[dflash2-large-vulkan]] for the archived runbook.
> The Q8_0 AGENT experiment was retired on 2026-09-01 after repeated
> `set_rows` crashes and device-local allocation failures. The former caf-eemin
> ROCmI4 embedded-MTP model and the v3 PromptForge payload are historical
> deployments. The intermediate DFlash2 Vulkan deployment was retired on
> 2026-08-21. See [[kingjones-q8agent-rocmfpx]] and [[dflash2-large-vulkan]].
## Validation