dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context

Browse Source 
frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache,
and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning
is on with preserve_thinking + preserve_reasoning in the chat kwargs.

- frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on,
  REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking,
  preserve_reasoning}
- serve-frank-large.sh: standalone draft-mtp profile (already switched off the
  Kairic/PromptForge path)
- download-q8agent.sh: reproducible HF fetch of the three payload files
- docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook
- AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note
- qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB
  (applied 2026-08-25), full 262k KV + Q8 payload fit in one slot
- frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled

The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.
This commit is contained in:
Frank
2026-08-25 07:12:02 -07:00
parent 24efa98d56
commit 54b2d84613
11 changed files with 350 additions and 129 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/index.md
+1
View File
@@ -99,6 +99,7 @@ The LLM inference fleet on the two hosts. Full index:
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[eemin-rocmi4-mtp]] — the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw (frank-default-large, active)
- [[kingjones-strix-mtp]] — the RETIRED kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (superseded 2026-08-25)
- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook for frank-default-large (active): serving, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)