Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context
frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache,
and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning
is on with preserve_thinking + preserve_reasoning in the chat kwargs.
- frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on,
REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking,
preserve_reasoning}
- serve-frank-large.sh: standalone draft-mtp profile (already switched off the
Kairic/PromptForge path)
- download-q8agent.sh: reproducible HF fetch of the three payload files
- docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook
- AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note
- qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB
(applied 2026-08-25), full 262k KV + Q8 payload fit in one slot
- frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled
The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.
This commit is contained in:
+8
-16
@@ -137,24 +137,16 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
|
||||
|
||||
### Dankclaw — frank-default-large
|
||||
|
||||
The active service on Dankclaw is the charlie12345/ROCmFPX llama.cpp runtime
|
||||
(with the opt-in `ROCmI4 W4A4` path) serving `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF`
|
||||
(`eemin` namespace, `Q4_0_ROCMI4`, embedded NextN/MTP head) under the alias
|
||||
`frank-default-large` on port 7770. It runs
|
||||
`/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server` (built
|
||||
against system ROCm 7.2.4 / amdclang 22) with a **strict Qwen MTP** draft
|
||||
(`--spec-type draft-mtp --spec-mtp-strict-qwen`) running on ROCm0 with all
|
||||
layers (`--spec-draft-ngl all`) — no separate draft file, because the MTP head
|
||||
is embedded in the main GGUF. K/V are f16, 65536 context (192K below the
|
||||
native 262144 maximum; 1 slot, concurrent requests queue, the server never
|
||||
preempts a running generation),
|
||||
`-t 16 -tb 32`, batch 512 / ubatch 256 (ubatch is swappable; sweep it on 8k
|
||||
context with `scripts/bench-ubatch-rocmi4.sh`), 16 context checkpoints, 8 GiB
|
||||
prompt cache, and reasoning in deepseek format. The unit file is
|
||||
`scripts/frank-default-large.service`. It launches through
|
||||
The active service on Dankclaw uses the ROCmFPX llama.cpp runtime to serve
|
||||
[kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF)
|
||||
under the alias `frank-default-large` on port 7770. The main model is
|
||||
`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
|
||||
`mtp-Qwen3.8-27B-Q4_0.gguf`, and the BF16 vision projector is loaded alongside
|
||||
it. The K-side KV cache remains `f16`; the V-side cache uses `turbo4`. The
|
||||
service keeps the existing 65536 context / 1-slot profile. The unit file is
|
||||
`scripts/frank-default-large.service`; it launches through
|
||||
`scripts/serve-frank-large.sh`, which reads tunables from
|
||||
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
|
||||
See [[eemin-rocmi4-mtp]].
|
||||
|
||||
> The kingjones ROCmFP4 STRIX MTP model served the slot from 2026-08-24; it
|
||||
> was replaced by the caf-eemin ROCmI4 embedded-MTP model on 2026-08-25. The
|
||||
|
||||
Reference in New Issue
Block a user