Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context

frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache,
and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning
is on with preserve_thinking + preserve_reasoning in the chat kwargs.

- frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on,
  REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking,
  preserve_reasoning}
- serve-frank-large.sh: standalone draft-mtp profile (already switched off the
  Kairic/PromptForge path)
- download-q8agent.sh: reproducible HF fetch of the three payload files
- docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook
- AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note
- qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB
  (applied 2026-08-25), full 262k KV + Q8 payload fit in one slot
- frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled

The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.
This commit is contained in:
Frank committed 2026-08-25 07:12:02 -07:00
1 parent 24efa98d56
commit 54b2d84613
11 files changed
+350 -129

No files matched your search

+8 -6
View File
@@ -19,7 +19,7 @@ Related pages:
| 2 | MTP `n_max` explicit 4–5 + Q4_0 draft head | +5–15% on the spec share | restart, 1.56 GB file | test 08-16 |
| 3 | Plain `Q8_0` A/B vs UD-Q8_K_XL (same 8 bits) | dequant ALUs may differ | restart, 29 GB file | test 08-16 |
| 4 | Prefill path: ROCm backend + tuned ubatch/threads-batch | 2–3× prefill for agentic turns | per-workload backend choice | partial data |
| 5 | TTM `pages_limit` raise (15.2 GiB → 90+) | removes a hard VRAM ceiling, stable 96 GiB budget | reboot | pending |
| 5 | TTM `pages_limit` raised to 114 GiB | removes the hard VRAM ceiling; the full 262k f16 KV plus the Q8 AGENT payload, MTP draft, and mmproj fit in one slot | modprobe, set live 2026-08-25 | applied (29,883,536 pages) |
| 6 | Vulkan ICD pinning + loader-layer disable | robustness; small | service env vars | test 08-16 |
Everything else (quantization downgrade, NVFP4, larger drafts) is documented
@@ -166,11 +166,13 @@ hybrid state, upstream). Levers:
### 5. OS level (from the Improvements doc, re-verified 2026-08-15)
- **TTM `pages_limit` is still 3,995,281 pages ≈ 15.24 GiB** — verified
today, unchanged. The 96 GiB GTT budget only materializes if the TTM cap
is raised (kernel parameter, needs reboot). Until then the effective
ceiling is lower and the 28 GiB load-stall risk remains. **Highest-value
single OS change; schedule with the next reboot.**
- **TTM `pages_limit` is now 114 GiB (29,883,536 pages)** — applied 2026-08-25.
Written live to `/sys/module/ttm/parameters/pages_limit` (no reboot required)
and persisted in `/etc/modprobe.d/ttm.conf`. The full 262k f16 KV cache plus
the ~28 GB Q8 AGENT payload, MTP draft, and mmproj now fit in one slot. The
load-stall risk from the old 15 GiB cap is resolved. Note: 96 GiB is the
driver-reported VRAM carve-out (VGM/SVM pool); the TTM pages_limit is the
kernel page map that was the real ceiling.
- `iommu=pt` — verified present on the kernel cmdline. Done.
- **Vulkan ICD pinning** — `radeon_icd.json` exists; no `VK_ICD_FILENAMES`
or loader-layer env is set. Pinning the radeon ICD and disabling loader