Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context
frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache,
and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning
is on with preserve_thinking + preserve_reasoning in the chat kwargs.
- frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on,
REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking,
preserve_reasoning}
- serve-frank-large.sh: standalone draft-mtp profile (already switched off the
Kairic/PromptForge path)
- download-q8agent.sh: reproducible HF fetch of the three payload files
- docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook
- AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note
- qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB
(applied 2026-08-25), full 262k KV + Q8 payload fit in one slot
- frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled
The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.
This commit is contained in:
1 parent
24efa98d56
commit
54b2d84613
11 files changed
+350
-129
No files matched your search
+10
-8
@@ -1,10 +1,10 @@
|
||||
# LLM Fleet — Knowledge Base
|
||||
|
||||
Documentation for the LLM inference fleet on two local machines
|
||||
(2026-08-24): upstream llama.cpp on Dankserver (frank-default-small,
|
||||
Ornith-1.5-35B-A3B CRACK) and CIRU ROCmFPX llama.cpp on Dankclaw
|
||||
(frank-default-large, kingjones Qwen3.8-27B ROCmFP4). Both services
|
||||
run the Qwen-Sharp chat template.
|
||||
(2026-09-01): upstream llama.cpp on Dankserver (frank-default-small,
|
||||
Ornith-1.5-35B-A3B CRACK) and ROCmFPX llama.cpp on Dankclaw
|
||||
(frank-default-large, kingjones Qwen3.8-27B ROCmFPX Q8 AGENT). Both
|
||||
services run the Qwen-Sharp chat template.
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -28,8 +28,8 @@ ngram-mod, selected experts offloaded via override, 491520 ctx]
|
||||
D[gfx1151 iGPU 64 GiB unified memory]
|
||||
E[CIRU ROCmFPX llama.cpp PromptForge v3
|
||||
frank-default-large, port 7770]
|
||||
F[Qwen3.8-27B-Uncensored-ROCmFP4 Q6_0 AGENT
|
||||
MTP draft + Sharp template, 262K ctx / 1 slot]
|
||||
F[Qwen3.8-27B ROCmFPX Q8_0 AGENT
|
||||
MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
|
||||
D --> E --> F
|
||||
end
|
||||
|
||||
@@ -48,8 +48,10 @@ MTP draft + Sharp template, 262K ctx / 1 slot]
|
||||
|
||||
- [[deployment]] — deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[eemin-rocmi4-mtp]] — the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model
|
||||
on Dankclaw (frank-default-large, active 2026-08-25)
|
||||
- [kingjones Qwen3.8-27B ROCmFPX Q8 AGENT](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF) — active frank-default-large model on Dankclaw; standalone MTP and Turbo4 V cache
|
||||
- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook: serving profile, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
|
||||
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
|
||||
on Dankclaw (retired)
|
||||
- [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
|
||||
Dankserver (frank-default-small, active 2026-08-24)
|
||||
- [[kingjones-strix-mtp]] — the retired kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
|
||||
|
||||
Reference in new issue
Block a user