Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context

frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache,
and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning
is on with preserve_thinking + preserve_reasoning in the chat kwargs.

- frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on,
  REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking,
  preserve_reasoning}
- serve-frank-large.sh: standalone draft-mtp profile (already switched off the
  Kairic/PromptForge path)
- download-q8agent.sh: reproducible HF fetch of the three payload files
- docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook
- AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note
- qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB
  (applied 2026-08-25), full 262k KV + Q8 payload fit in one slot
- frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled

The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.
This commit is contained in:
Frank
2026-08-25 07:12:02 -07:00
parent 24efa98d56
commit 54b2d84613
11 changed files with 350 additions and 129 deletions
+23 -6
View File
@@ -27,14 +27,17 @@ host.
| Target | Runtime | Service | Model | Port |
|--------|---------|---------|-------|------|
| Dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
| Dankclaw | charlie12345/ROCmFPX llama.cpp, ROCmI4 W4A4 runtime | frank-default-large | caf-eemin Qwen3.8-27B-ROCmI4-MTP Q4_0_ROCMI4 (embedded NextN/MTP head, gfx1151 IU4 batched verification), 262K context / 1 slot | 7770 |
| Dankclaw | charlie12345/ROCmFPX llama.cpp | frank-default-large | kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT Q8_0_ROCMFPX (standalone MTP draft head, Turbo4 V cache, 262K context / 1 slot), reasoning on | 7770 |
History: the 2026-08-17 redo retired the pipeline-parallel split and the
FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
Dankclaw. On 2026-08-21 a DFlash2 Vulkan deployment briefly replaced CIRU,
then was reverted the same day for speed. On 2026-08-22 the service briefly
swapped to the Kairic Edge IU4 runtime, then was reverted to v3 the same day
(operator decision). Runbooks: [docs/LLM/index.md](docs/LLM/index.md).
(operator decision). On 2026-08-25 the service switched to the kingjones
Qwen3.8-27B-ROCmFPX-Q8_0-AGENT model (standalone MTP draft, 262144 context,
reasoning on); the previous Kairic Edge model was retired the same day. Runbooks:
[docs/LLM/index.md](docs/LLM/index.md).
Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`.
## Repository Layout
@@ -179,7 +182,21 @@ When you rename a service or move a model to a new host, update
## Runtime & Troubleshooting
### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, ACTIVE)
### kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT llama.cpp (Dankclaw, ACTIVE)
The large service serves `kingjones/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT`
(public) from `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/`
with a charlie12345/ROCmFPX `llama-server` (build `build-kairic`). It runs a
standalone MTP draft head on ROCm0, f16 K + TurboQuant V KV cache, a full
262144 context / 1 slot, and reasoning on (Sharp template, budget -1,
`--reasoning-format none`; chat kwargs keep `preserve_thinking` +
`preserve_reasoning`). The payload (~28 GB main + 1.7 GB draft + 0.9 GB
mmproj, ~30 GB total) plus the 262k KV cache fit in the lifted 114 GiB TTM/GTT
budget (raised 2026-08-25). Env `scripts/frank-default-large.env`, serve
`scripts/serve-frank-large.sh`, runbook
[docs/LLM/kingjones-q8agent-rocmfpx.md](docs/LLM/kingjones-q8agent-rocmfpx.md).
### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, prior v3, 2026-08-18 to 2026-08-25)
The v4 runtime serves `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (`eemin` namespace,
public) from `/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server`
@@ -300,9 +317,9 @@ graph TD
- Never run `pkill -f` with a pattern that appears in your own command line.
It kills your ssh session. Use `pkill -x <exact-name>`.
- Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM
(`--no-mmap`, MoE experts on CPU) in a few minutes cold. Dankclaw v3
loads the ~36.7 GiB five-file payload in ~18 s with a warm page cache,
a few minutes cold. Health returns 503 while loading. Do not kill the
(`--no-mmap`, MoE experts on CPU) in a few minutes cold. Dankclaw loads the
~30 GiB Q8_0 AGENT payload (main + MTP draft + mmproj) a few minutes cold
with a fresh KV cache; ~18 s with a warm page cache. Health returns 503 while loading. Do not kill the
process. `deploy.sh` waits up to 20 minutes on both targets (see
`LLM_WAIT`).
- A client abort and a server release look different in the journal. A