Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context
frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache,
and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning
is on with preserve_thinking + preserve_reasoning in the chat kwargs.
- frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on,
REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking,
preserve_reasoning}
- serve-frank-large.sh: standalone draft-mtp profile (already switched off the
Kairic/PromptForge path)
- download-q8agent.sh: reproducible HF fetch of the three payload files
- docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook
- AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note
- qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB
(applied 2026-08-25), full 262k KV + Q8 payload fit in one slot
- frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled
The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.
This commit is contained in:
1 parent
24efa98d56
commit
54b2d84613
11 files changed
+350
-129
No files matched your search
@@ -19,7 +19,7 @@ Related pages:
|
||||
| 2 | MTP `n_max` explicit 4–5 + Q4_0 draft head | +5–15% on the spec share | restart, 1.56 GB file | test 08-16 |
|
||||
| 3 | Plain `Q8_0` A/B vs UD-Q8_K_XL (same 8 bits) | dequant ALUs may differ | restart, 29 GB file | test 08-16 |
|
||||
| 4 | Prefill path: ROCm backend + tuned ubatch/threads-batch | 2–3× prefill for agentic turns | per-workload backend choice | partial data |
|
||||
| 5 | TTM `pages_limit` raise (15.2 GiB → 90+) | removes a hard VRAM ceiling, stable 96 GiB budget | reboot | pending |
|
||||
| 5 | TTM `pages_limit` raised to 114 GiB | removes the hard VRAM ceiling; the full 262k f16 KV plus the Q8 AGENT payload, MTP draft, and mmproj fit in one slot | modprobe, set live 2026-08-25 | applied (29,883,536 pages) |
|
||||
| 6 | Vulkan ICD pinning + loader-layer disable | robustness; small | service env vars | test 08-16 |
|
||||
|
||||
Everything else (quantization downgrade, NVFP4, larger drafts) is documented
|
||||
@@ -166,11 +166,13 @@ hybrid state, upstream). Levers:
|
||||
|
||||
### 5. OS level (from the Improvements doc, re-verified 2026-08-15)
|
||||
|
||||
- **TTM `pages_limit` is still 3,995,281 pages ≈ 15.24 GiB** — verified
|
||||
today, unchanged. The 96 GiB GTT budget only materializes if the TTM cap
|
||||
is raised (kernel parameter, needs reboot). Until then the effective
|
||||
ceiling is lower and the 28 GiB load-stall risk remains. **Highest-value
|
||||
single OS change; schedule with the next reboot.**
|
||||
- **TTM `pages_limit` is now 114 GiB (29,883,536 pages)** — applied 2026-08-25.
|
||||
Written live to `/sys/module/ttm/parameters/pages_limit` (no reboot required)
|
||||
and persisted in `/etc/modprobe.d/ttm.conf`. The full 262k f16 KV cache plus
|
||||
the ~28 GB Q8 AGENT payload, MTP draft, and mmproj now fit in one slot. The
|
||||
load-stall risk from the old 15 GiB cap is resolved. Note: 96 GiB is the
|
||||
driver-reported VRAM carve-out (VGM/SVM pool); the TTM pages_limit is the
|
||||
kernel page map that was the real ceiling.
|
||||
- `iommu=pt` — verified present on the kernel cmdline. Done.
|
||||
- **Vulkan ICD pinning** — `radeon_icd.json` exists; no `VK_ICD_FILENAMES`
|
||||
or loader-layer env is set. Pinning the radeon ICD and disabling loader
|
||||
|
||||
Reference in new issue
Block a user