Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context
frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache,
and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning
is on with preserve_thinking + preserve_reasoning in the chat kwargs.
- frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on,
REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking,
preserve_reasoning}
- serve-frank-large.sh: standalone draft-mtp profile (already switched off the
Kairic/PromptForge path)
- download-q8agent.sh: reproducible HF fetch of the three payload files
- docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook
- AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note
- qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB
(applied 2026-08-25), full 262k KV + Q8 payload fit in one slot
- frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled
The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.
This commit is contained in:
1 parent
24efa98d56
commit
54b2d84613
11 files changed
+350
-129
No files matched your search
@@ -21,20 +21,24 @@ marked "verify first". Re-verified 2026-08-15/16; corrections marked below.
|
||||
- **KV cache**: production runs `q8_0` K **and** `q8_0` V (both quantized).
|
||||
- **Spec**: DFlash/DSpark are retired. Production is `--spec-default`
|
||||
(ngram-mod) with draft-mtp first. See [[qwen3.8-spec-matrix]].
|
||||
- **TTM**: still 15.24 GiB as of 2026-08-16 — the modprobe fix is still
|
||||
pending a reboot.
|
||||
- **TTM**: raised to 114 GiB on 2026-08-25, set live (no reboot) and
|
||||
persisted in `/etc/modprobe.d/ttm.conf`. The effective VRAM ceiling that
|
||||
once held the service at ~15 GiB is gone.
|
||||
|
||||
## Verified Memory Picture (measured 2026-08-11)
|
||||
|
||||
| Figure | Value | Source |
|
||||
| ------ | ----- | ------ |
|
||||
| OS physical RAM | 30.5 GiB | `/proc/meminfo` |
|
||||
| TTM pages_limit | 15.24 GiB (3,995,281 x 4KiB) | `/sys/module/ttm/parameters/pages_limit` |
|
||||
| TTM pages_limit | 114 GiB (29,883,536 x 4KiB) — raised 2026-08-25 | `/sys/module/ttm/parameters/pages_limit` |
|
||||
| ROCm VRAM total | 96.0 GiB | `rocm-smi --showmeminfo vram` |
|
||||
| ROCm VRAM used | 76.2 GiB | `rocm-smi` while serving 3x200700 |
|
||||
|
||||
The OS accounts only ~30.5 GiB physical RAM, and the TTM GPU-addressable pool
|
||||
is capped at ~15.24 GiB. ROCm/SMI still reports a 96 GiB VRAM carve-out (VGM).
|
||||
The OS accounts only ~30.5 GiB physical RAM. The TTM GPU-addressable pool
|
||||
was capped at ~15.24 GiB (the real pre-2026-08-25 ceiling); that cap is now
|
||||
lifted to 114 GiB. ROCm/SMI still reports a 96 GiB VRAM carve-out (VGM) — a
|
||||
different, driver-reported figure that has always been larger than the kernel
|
||||
cap.
|
||||
This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
|
||||
limits documented in [[frank-default-large-benchmark]].
|
||||
|
||||
@@ -77,7 +81,7 @@ All require root and a reboot for the modprobe change.
|
||||
|
||||
| Setting | Action | Effect | Priority |
|
||||
| ------- | ------ | ------ | -------- |
|
||||
| TTM GPU pool | Set `amd-ttm --set 96` (write `/etc/modprobe.d/ttm.conf`), reboot | Lifts the 15 GiB TTM cap toward full RAM; more headroom for GPU buffers | High |
|
||||
| TTM GPU pool | `/etc/modprobe.d/ttm.conf` → `options ttm pages_limit=29883536 page_pool_size=29883536` (applied 2026-08-25, set live then persisted) | Lifts the 15 GiB cap to 114 GiB; the full 262k KV + Q8 AGENT payload fit in one slot | Done |
|
||||
| Power profile | **N/A on this machine** (2026-08-15): no `power_profile` sysfs; GPU DPM already `performance` | none | ~~Medium~~ |
|
||||
| NVMe scheduler | Already `none` (`[none]` confirmed) | No change | Done |
|
||||
| BIOS VGM | Set Variable Graphics Memory / UMA to "High" in the EVO-X2 BIOS | Max dedicated VRAM for the GPU carve-out; verify in the live BIOS | Verify |
|
||||
|
||||
Reference in new issue
Block a user