dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

Restore Q6 AGENT large model

Browse Source
This commit is contained in:
Frank
2026-08-25 13:36:07 -07:00
parent 205b11aae4
commit 86d5f38142
8 changed files with 51 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/LLM/deployment.md
+10 -10
View File
@@ -18,7 +18,7 @@ repository to a running service.
| Target | Host | Runtime | Service | Port |
|--------|------|---------|---------|------|
| Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q6_K, native MTP draft, selected MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (caf-eemin Qwen3.8-27B ROCmI4 Q4_0, embedded MTP, ROCmI4 W4A4, 262K ctx / 1 slot) | frank-default-large | 7770 |
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (kingjones Qwen3.8 Uncensored Q6_0 ROCmFPX AGENT, standalone MTP, 262K ctx / 1 slot) | frank-default-large | 7770 |
## Workflow
@@ -138,21 +138,21 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
### Dankclaw — frank-default-large
The active service on Dankclaw uses the ROCmFPX llama.cpp runtime to serve
[kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF)
[kingjones777/Qwen3.8-Uncensored-ROCmFP4-STRIX-MTP-GGUF](https://huggingface.co/kingjones777/Qwen3.8-Uncensored-ROCmFP4-STRIX-MTP-GGUF)
under the alias `frank-default-large` on port 7770. The main model is
`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the BF16 vision projector is loaded alongside
`Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the Q8_0 vision projector is loaded alongside
it. The K-side KV cache remains `f16`; the V-side cache uses `turbo4`. The
service keeps the existing 65536 context / 1-slot profile. The unit file is
service uses the 262144 context / 1-slot profile. The unit file is
`scripts/frank-default-large.service`; it launches through
`scripts/serve-frank-large.sh`, which reads tunables from
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
> The kingjones ROCmFP4 STRIX MTP model served the slot from 2026-08-24; it
> was replaced by the caf-eemin ROCmI4 embedded-MTP model on 2026-08-25. The
> v3 PromptForge payload and the CIRU ROCmFPX runtime are retired on this slot.
> The intermediate DFlash2 Vulkan deployment was retired the same day (2026-08-21).
> See [[dflash2-large-vulkan]] for the archived runbook.
> The Q8_0 AGENT experiment was retired on 2026-09-01 after repeated
> `set_rows` crashes and device-local allocation failures. The former caf-eemin
> ROCmI4 embedded-MTP model and the v3 PromptForge payload are historical
> deployments. The intermediate DFlash2 Vulkan deployment was retired on
> 2026-08-21. See [[kingjones-q8agent-rocmfpx]] and [[dflash2-large-vulkan]].
## Validation
docs/LLM/index.md
+7 -6
View File
@@ -3,7 +3,7 @@
Documentation for the LLM inference fleet on two local machines
(2026-09-01): upstream llama.cpp on Dankserver (frank-default-small,
Ornith-1.5-35B-A3B CRACK) and ROCmFPX llama.cpp on Dankclaw
(frank-default-large, kingjones Qwen3.8-27B ROCmFPX Q8 AGENT). Both
(frank-default-large, kingjones Qwen3.8 Uncensored Q6 AGENT). Both
services run the Qwen-Sharp chat template.
> 2026-08-22: the former `vllmScripts` repository is merged into the
@@ -28,7 +28,7 @@ ngram-mod, selected experts offloaded via override, 491520 ctx]
D[gfx1151 iGPU 64 GiB unified memory]
E[charlie12345/ROCmFPX build-kairic (v81)
frank-default-large, port 7770]
F[Qwen3.8-27B ROCmFPX Q8_0 AGENT
F[Qwen3.8 Uncensored Q6 ROCmFPX AGENT
MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
D --> E --> F
end
@@ -48,14 +48,15 @@ MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [kingjones Qwen3.8-27B ROCmFPX Q8 AGENT](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF) — active frank-default-large model on Dankclaw; standalone MTP and Turbo4 V cache
- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook: serving profile, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
- [kingjones Qwen3.8 Uncensored ROCmFP4 STRIX MTP](https://huggingface.co/kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF) — active frank-default-large model on Dankclaw; Q6 AGENT main model with standalone MTP
- [[kingjones-strix-mtp]] — the active Q6 AGENT runbook: serving profile, standalone MTP, reasoning + preserve_thinking, 114 GiB TTM/GTT
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
on Dankclaw (retired)
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
Dankserver (frank-default-small, active 2026-08-24)
- [[kingjones-strix-mtp]] — the retired kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
model on Dankclaw (superseded 2026-08-25 by the ROCmI4 embedded-MTP model)
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
model on Dankclaw (restored 2026-09-01 after the Q8_0 AGENT experiment)
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
Dankserver (frank-default-small, active 2026-08-24)
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime
docs/LLM/kingjones-strix-mtp.md
+5 -6
View File
@@ -15,13 +15,12 @@ timestamp: 2026-08-24
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
> **Retired 2026-08-25.** The kingjones ROCmFP4 STRIX MTP payload has been
> replaced by the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on the same
> port and iGPU, under a new runtime (charlie12345/ROCmFPX ROCmI4 W4A4). See
> [[eemin-rocmi4-mtp]]. The old kingjones GGUF + standalone MTP Q4_0 draft are
> retired on this slot. The CIRU ROCmFPX runtime/binary is also retired here.
> **Active again 2026-09-01.** The kingjones ROCmFP4 STRIX MTP payload is
> restored as `frank-default-large` after the Q8_0 AGENT experiment. The Q8_0
> model was retired because it crashed before serving. The Q6 AGENT model,
> standalone MTP draft, Q8 projector, and PromptForge runtime are now active.
> _This page documents the retired model and its runbook for reference._
> This page documents the active model and its runbook.
`frank-default-large` (port 7770) serves
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
docs/index.md
+3 -3
View File
@@ -97,9 +97,9 @@ The LLM inference fleet on the two hosts. Full index:
- [[deployment]] — unified deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[eemin-rocmi4-mtp]] — the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw (frank-default-large, active)
- [[kingjones-strix-mtp]] — the RETIRED kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (superseded 2026-08-25)
- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook for frank-default-large (active): serving, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
- [[kingjones-strix-mtp]] — the active kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (restored 2026-09-01)
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)