Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context

frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache,
and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning
is on with preserve_thinking + preserve_reasoning in the chat kwargs.

- frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on,
  REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking,
  preserve_reasoning}
- serve-frank-large.sh: standalone draft-mtp profile (already switched off the
  Kairic/PromptForge path)
- download-q8agent.sh: reproducible HF fetch of the three payload files
- docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook
- AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note
- qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB
  (applied 2026-08-25), full 262k KV + Q8 payload fit in one slot
- frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled

The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.

docs/LLM/Dankclaw Improvements.md

@@ -21,20 +21,24 @@ marked "verify first". Re-verified 2026-08-15/16; corrections marked below.
 - **KV cache**: production runs `q8_0` K **and** `q8_0` V (both quantized).
 - **Spec**: DFlash/DSpark are retired. Production is `--spec-default`
   (ngram-mod) with draft-mtp first. See [[qwen3.8-spec-matrix]].
-- **TTM**: still 15.24 GiB as of 2026-08-16 — the modprobe fix is still
-  pending a reboot.
+- **TTM**: raised to 114 GiB on 2026-08-25, set live (no reboot) and
+  persisted in `/etc/modprobe.d/ttm.conf`. The effective VRAM ceiling that
+  once held the service at ~15 GiB is gone.
 
 ## Verified Memory Picture (measured 2026-08-11)
 
 | Figure | Value | Source |
 | ------ | ----- | ------ |
 | OS physical RAM | 30.5 GiB | `/proc/meminfo` |
-| TTM pages_limit | 15.24 GiB (3,995,281 x 4KiB) | `/sys/module/ttm/parameters/pages_limit` |
+| TTM pages_limit | 114 GiB (29,883,536 x 4KiB) — raised 2026-08-25 | `/sys/module/ttm/parameters/pages_limit` |
 | ROCm VRAM total | 96.0 GiB | `rocm-smi --showmeminfo vram` |
 | ROCm VRAM used | 76.2 GiB | `rocm-smi` while serving 3x200700 |
 
-The OS accounts only ~30.5 GiB physical RAM, and the TTM GPU-addressable pool
-is capped at ~15.24 GiB. ROCm/SMI still reports a 96 GiB VRAM carve-out (VGM).
+The OS accounts only ~30.5 GiB physical RAM. The TTM GPU-addressable pool
+was capped at ~15.24 GiB (the real pre-2026-08-25 ceiling); that cap is now
+lifted to 114 GiB. ROCm/SMI still reports a 96 GiB VRAM carve-out (VGM) — a
+different, driver-reported figure that has always been larger than the kernel
+cap.
 This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
 limits documented in [[frank-default-large-benchmark]].
 
@@ -77,7 +81,7 @@ All require root and a reboot for the modprobe change.
 
 | Setting | Action | Effect | Priority |
 | ------- | ------ | ------ | -------- |
-| TTM GPU pool | Set `amd-ttm --set 96` (write `/etc/modprobe.d/ttm.conf`), reboot | Lifts the 15 GiB TTM cap toward full RAM; more headroom for GPU buffers | High |
+| TTM GPU pool | `/etc/modprobe.d/ttm.conf` → `options ttm pages_limit=29883536 page_pool_size=29883536` (applied 2026-08-25, set live then persisted) | Lifts the 15 GiB cap to 114 GiB; the full 262k KV + Q8 AGENT payload fit in one slot | Done |
 | Power profile | **N/A on this machine** (2026-08-15): no `power_profile` sysfs; GPU DPM already `performance` | none | ~~Medium~~ |
 | NVMe scheduler | Already `none` (`[none]` confirmed) | No change | Done |
 | BIOS VGM | Set Variable Graphics Memory / UMA to "High" in the EVO-X2 BIOS | Max dedicated VRAM for the GPU carve-out; verify in the live BIOS | Verify |

docs/LLM/deployment.md

@@ -137,24 +137,16 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
 
 ### Dankclaw — frank-default-large
 
-The active service on Dankclaw is the charlie12345/ROCmFPX llama.cpp runtime
-(with the opt-in `ROCmI4 W4A4` path) serving `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF`
-(`eemin` namespace, `Q4_0_ROCMI4`, embedded NextN/MTP head) under the alias
-`frank-default-large` on port 7770. It runs
-`/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server` (built
-against system ROCm 7.2.4 / amdclang 22) with a **strict Qwen MTP** draft
-(`--spec-type draft-mtp --spec-mtp-strict-qwen`) running on ROCm0 with all
-layers (`--spec-draft-ngl all`) — no separate draft file, because the MTP head
-is embedded in the main GGUF. K/V are f16, 65536 context (192K below the
-native 262144 maximum; 1 slot, concurrent requests queue, the server never
-preempts a running generation),
-`-t 16 -tb 32`, batch 512 / ubatch 256 (ubatch is swappable; sweep it on 8k
-context with `scripts/bench-ubatch-rocmi4.sh`), 16 context checkpoints, 8 GiB
-prompt cache, and reasoning in deepseek format. The unit file is
-`scripts/frank-default-large.service`. It launches through
+The active service on Dankclaw uses the ROCmFPX llama.cpp runtime to serve
+[kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF)
+under the alias `frank-default-large` on port 7770. The main model is
+`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
+`mtp-Qwen3.8-27B-Q4_0.gguf`, and the BF16 vision projector is loaded alongside
+it. The K-side KV cache remains `f16`; the V-side cache uses `turbo4`. The
+service keeps the existing 65536 context / 1-slot profile. The unit file is
+`scripts/frank-default-large.service`; it launches through
 `scripts/serve-frank-large.sh`, which reads tunables from
 `scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
-See [[eemin-rocmi4-mtp]].
 
 > The kingjones ROCmFP4 STRIX MTP model served the slot from 2026-08-24; it
 > was replaced by the caf-eemin ROCmI4 embedded-MTP model on 2026-08-25. The

docs/LLM/index.md

@@ -1,10 +1,10 @@
 # LLM Fleet — Knowledge Base
 
 Documentation for the LLM inference fleet on two local machines
-(2026-08-24): upstream llama.cpp on Dankserver (frank-default-small,
-Ornith-1.5-35B-A3B CRACK) and CIRU ROCmFPX llama.cpp on Dankclaw
-(frank-default-large, kingjones Qwen3.8-27B ROCmFP4). Both services
-run the Qwen-Sharp chat template.
+(2026-09-01): upstream llama.cpp on Dankserver (frank-default-small,
+Ornith-1.5-35B-A3B CRACK) and ROCmFPX llama.cpp on Dankclaw
+(frank-default-large, kingjones Qwen3.8-27B ROCmFPX Q8 AGENT). Both
+services run the Qwen-Sharp chat template.
 
 > 2026-08-22: the former `vllmScripts` repository is merged into the
 > serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -28,8 +28,8 @@ ngram-mod, selected experts offloaded via override, 491520 ctx]
         D[gfx1151 iGPU 64 GiB unified memory]
         E[CIRU ROCmFPX llama.cpp PromptForge v3
 frank-default-large, port 7770]
-        F[Qwen3.8-27B-Uncensored-ROCmFP4 Q6_0 AGENT
-MTP draft + Sharp template, 262K ctx / 1 slot]
+        F[Qwen3.8-27B ROCmFPX Q8_0 AGENT
+MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
         D --> E --> F
     end
 
@@ -48,8 +48,10 @@ MTP draft + Sharp template, 262K ctx / 1 slot]
 
 - [[deployment]] — deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
-- [[eemin-rocmi4-mtp]] — the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model
-  on Dankclaw (frank-default-large, active 2026-08-25)
+- [kingjones Qwen3.8-27B ROCmFPX Q8 AGENT](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF) — active frank-default-large model on Dankclaw; standalone MTP and Turbo4 V cache
+- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook: serving profile, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
+- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
+  on Dankclaw (retired)
 - [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
   Dankserver (frank-default-small, active 2026-08-24)
 - [[kingjones-strix-mtp]] — the retired kingjones Qwen3.8-27B ROCmFP4 STRIX MTP

docs/LLM/kingjones-q8agent-rocmfpx.md

@@ -0,0 +1,165 @@
+# frank-default-large: kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
+
+**Status:** ACTIVE on Dankclaw (since 2026-08-25).
+
+**Alias:** `frank-default-large`, port **7770**.
+
+The large service serves the kingjones `Qwen3.8-27B-ROCmFPX-Q8_0-AGENT` GGUF
+from `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/` with a
+charlie12345/ROCmFPX `llama-server` (build `build-kairic`). It runs a
+**standalone MTP draft head** (loaded separately, on ROCm0), **TurboQuant** V
+KV cache, the **Sharp** chat template with **reasoning on**, and a **full
+262144 context / 1 slot** profile. The whole payload plus the 262k KV cache
+fit in the lifted **114 GiB TTM/GTT** budget (see the 114 GiB change in
+[[Dankclaw Improvements]] and [[qwen3.8-strix-optimizations]]).
+
+| Figure | Value |
+| ------ | ----- |
+| Model | kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT (`Q8_0_ROCMFPX`) |
+| HF repo | `kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF` (public) |
+| Runtime | charlie12345/ROCmFPX `llama-server`, `build-kairic` |
+| Draft head | standalone `mtp-Qwen3.8-27B-Q4_0.gguf` on ROCm0 |
+| KV cache | f16 K, TurboQuant (`turbo4`) V, full 262144 context, 1 slot |
+| Reasoning | on, `deepseek`, budget `-1`, `--reasoning-format none` |
+| Chat template | Sharp (repo-tracked), `enable_thinking` + `preserve_thinking` on |
+| Host path | `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/` |
+| Tunables | `scripts/frank-default-large.env` |
+| Serve script | `scripts/serve-frank-large.sh` |
+
+## Model files
+
+Three files, from the HF repo above:
+
+| File | Size | Purpose |
+| ---- | ---- | ------- |
+| `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` | 28.21 GB | main weights |
+| `mtp-Qwen3.8-27B-Q4_0.gguf` | 1.68 GB | standalone MTP draft head (loaded by the script) |
+| `mmproj-Qwen3.8-27B-BF16.gguf` | 0.93 GB | vision projector (`--mmproj`) |
+
+The repo also has a `mtp-Qwen3.8-27B-Q8_0.gguf` (3.16 GB) draft; the env uses
+the Q4_0 draft. Download them from the resolve URLs below (they are large —
+use the token in `~/.cache/huggingface/token`).
+
+## Download
+
+```bash
+# on Dankclaw
+TOKEN=$(cat ~/.cache/huggingface/token)
+REPO=kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF
+DIR=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT
+mkdir -p "$DIR"; cd "$DIR"
+dl(){ curl -fsSL -C - -o "$1" --retry 5 --retry-delay 3 --retry-max-time 3600 \
+  -H "Authorization: Bearer $TOKEN" \
+  "https://huggingface.co/$REPO/resolve/main/$1"; echo "done: $1"; }
+dl Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf
+dl mmproj-Qwen3.8-27B-BF16.gguf
+dl mtp-Qwen3.8-27B-Q4_0.gguf
+```
+
+The serve script checks each artifact is present and readable before launch
+(`scripts/frank-default-large.env` -> the four `MODEL_*` files plus the chat
+template). A missing file fails the artifact check with `missing or
+unreadable required artifact`.
+
+## Serving profile
+
+`scripts/serve-frank-large.sh` builds the `llama-server` call from the env:
+
+```
+/dev/llama-server -m <main> --alias frank-default-large
+  --host 0.0.0.0 --port 7770 --jinja
+  -dev ROCm0 -ngl 999 -np 1 -c 262144
+  -b 512 -ub 512 -fa on
+  -ctk f16 -ctv turbo4 -t 16 -tb 32
+  -ctxcp 32 --cache-ram 8192 --cache-prompt --cache-idle-slots --metrics
+  --spec-type draft-mtp --model-draft <mtp-Q4_0>
+  --mmproj <mmproj-BF16>
+  --spec-draft-device ROCm0 --spec-draft-ngl 999
+  --spec-draft-type-k f16 --spec-draft-type-v turbo4
+  --spec-draft-n-max 4 --spec-draft-n-min 0
+  --spec-draft-p-min 0.0 --spec-draft-p-split 0.10
+  --spec-draft-backend-sampling
+  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0
+  --presence-penalty 1.5
+  --reasoning on --reasoning-format none --reasoning-budget -1
+```
+
+Key choices:
+
+- **Standalone MTP draft** (`--model-draft`, `--spec-type draft-mtp`). The
+  AGENT GGUF has no embedded draft head; the draft head is a separate file on
+  ROCm0. This matches the Qwen3.8-27B AGENT shape (`--spec-draft-type-k
+  f16 --spec-draft-type-v turbo4`).
+- **f16 K + TurboQuant V** cache. f16 K is the safe long-ctx default; TurboQuant
+  halves the V cache at 262k while keeping quality (see the ubatch/threads-batch
+  notes in [[qwen3.8-strix-optimizations]]).
+- **Full 262144 context, 1 slot.** The whole context KV cache plus the ~30 GB
+  payload (main + draft + mmproj) fit in the 114 GiB budget. Concurrent
+  requests queue; the server never preempts a running generation.
+- **Reasoning on.** The Sharp template drives thinking (medium effort);
+  `--reasoning-format none` lets the template project the reasoning content into
+  `reasoning_content`. `--reasoning-budget -1` = no budget cap.
+
+## Reasoning + chat template
+
+The Sharp template (`scripts/chat-template.jinja`, repo-tracked) is a
+Qwen-AGENT shape. Env `CHAT_TEMPLATE_KWARGS` keeps the reasoning trail in the
+conversation:
+
+```json
+{"enable_thinking": true, "preserve_thinking": true, "preserve_reasoning": true}
+```
+
+`preserve_thinking: true` keeps the model's prior thinking in the next prompt
+(the user's "preserve thinking" request). `preserve_reasoning: true` keeps the
+model's prior reasoning trace across requests too (OpenWebUI sends
+`preserve_reasoning`/`preserve_thinking: true` when the conversation needs the
+full trail). For multi-turn reasoning, the template stays in the reasoning
+shape and `enable_thinking: true` per request.
+
+## Memory: the 114 GiB TTM/GTT requirement
+
+The 262k f16 KV cache is large (~a few GB) and the payload is ~30 GB. Both live
+in the Strix Halo unified-memory GTT. The TTM `pages_limit` was raised to
+114 GiB on 2026-08-25 (written live to
+`/sys/module/ttm/parameters/pages_limit`, then persisted in
+`/etc/modprobe.d/ttm.conf`). Without the raise the effective ceiling is ~15 GiB
+and a 28 GB + 262k-KV load stalls. The GTT page-limit change is per the
+`qwen3.8-strix-optimizations` table item 5.
+
+## Health + troubleshooting
+
+- Health: `curl http://192.168.0.143:7770/health` → `{"status":"ok"}`.
+- `/api/models` reports `n_ctx:262144, n_ctx_train:262144, n_params:27.3B`.
+- Model load is slow (a few minutes cold for the ~30 GB payload with a fresh
+  KV cache). Health returns 503 while loading. Do not kill the process.
+  `deploy.sh` waits up to 20 minutes (see `LLM_WAIT`).
+- Kernel spam about `amdgpu: SVM mapping failed` is the known documented
+  symptom, not root cause (see the crash-loop procedure in the project
+  AGENTS.md). Investigate other causes before assuming memory pressure.
+- A client abort logs `W srv stop: cancel task, id_task = N`; a natural stop
+  logs only `slot release` with `truncated = 0`.
+
+## Env tunables
+
+`scripts/frank-default-large.env`. Serving: `CTX_SIZE=262144`,
+`PARALLEL=1`, `BATCH_SIZE=512`, `UBATCH_SIZE=512`, `NUM_THREADS=16`,
+`THREAD_BATCH=32`, `CACHE_TYPE_K=f16`, `CACHE_TYPE_V=turbo4`,
+`CTX_CHECKPOINTS=32`, `CACHE_RAM=8192`. Reasoning: `REASONING=on`,
+`REASONING_FORMAT=none`. Sampling: `TEMPERATURE=0.7`, `TOP_K=20`,
+`TOP_P=0.8`. MTP: `MTP_N_MAX=4`, `MTP_N_MIN=0`, `MTP_P_MIN=0.0`,
+`MTP_P_SPLIT=0.10`. Strix Halo: `HSA_OVERRIDE_GFX_VERSION=11.5.1`,
+`GGML_HIP_ENABLE_UNIFIED_MEMORY=1`.
+
+**Never name a tunable `TEMP`, `HOST`, or `GPU_MAX_HEAP_SIZE`** — the ROCm HIP
+runtime reads those bare vars (see the 2026-08-13 TEMP incident). llama.cpp
+reads `--host/--port/--temp` as CLI flags, never as env vars.
+
+## History
+
+- **2026-08-25 (today):** switched the large service to the kingjones
+  Q8_0-AGENT model with 262144 context and reasoning on. Downloaded the three
+  payload files from the HF repo, raised the TTM/GTT to 114 GiB for the full
+  context, and persisted the env/serve script in this repo.
+- Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
+  retired the same day for the AGENT model). See [[kairic-edge]].

docs/LLM/qwen3.8-strix-optimizations.md

@@ -19,7 +19,7 @@ Related pages:
 | 2 | MTP `n_max` explicit 4–5 + Q4_0 draft head | +5–15% on the spec share | restart, 1.56 GB file | test 08-16 |
 | 3 | Plain `Q8_0` A/B vs UD-Q8_K_XL (same 8 bits) | dequant ALUs may differ | restart, 29 GB file | test 08-16 |
 | 4 | Prefill path: ROCm backend + tuned ubatch/threads-batch | 2–3× prefill for agentic turns | per-workload backend choice | partial data |
-| 5 | TTM `pages_limit` raise (15.2 GiB → 90+) | removes a hard VRAM ceiling, stable 96 GiB budget | reboot | pending |
+| 5 | TTM `pages_limit` raised to 114 GiB | removes the hard VRAM ceiling; the full 262k f16 KV plus the Q8 AGENT payload, MTP draft, and mmproj fit in one slot | modprobe, set live 2026-08-25 | applied (29,883,536 pages) |
 | 6 | Vulkan ICD pinning + loader-layer disable | robustness; small | service env vars | test 08-16 |
 
 Everything else (quantization downgrade, NVFP4, larger drafts) is documented
@@ -166,11 +166,13 @@ hybrid state, upstream). Levers:
 
 ### 5. OS level (from the Improvements doc, re-verified 2026-08-15)
 
-- **TTM `pages_limit` is still 3,995,281 pages ≈ 15.24 GiB** — verified
-  today, unchanged. The 96 GiB GTT budget only materializes if the TTM cap
-  is raised (kernel parameter, needs reboot). Until then the effective
-  ceiling is lower and the 28 GiB load-stall risk remains. **Highest-value
-  single OS change; schedule with the next reboot.**
+- **TTM `pages_limit` is now 114 GiB (29,883,536 pages)** — applied 2026-08-25.
+  Written live to `/sys/module/ttm/parameters/pages_limit` (no reboot required)
+  and persisted in `/etc/modprobe.d/ttm.conf`. The full 262k f16 KV cache plus
+  the ~28 GB Q8 AGENT payload, MTP draft, and mmproj now fit in one slot. The
+  load-stall risk from the old 15 GiB cap is resolved. Note: 96 GiB is the
+  driver-reported VRAM carve-out (VGM/SVM pool); the TTM pages_limit is the
+  kernel page map that was the real ceiling.
 - `iommu=pt` — verified present on the kernel cmdline. Done.
 - **Vulkan ICD pinning** — `radeon_icd.json` exists; no `VK_ICD_FILENAMES`
   or loader-layer env is set. Pinning the radeon ICD and disabling loader

docs/index.md

@@ -99,6 +99,7 @@ The LLM inference fleet on the two hosts. Full index:
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
 - [[eemin-rocmi4-mtp]] — the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw (frank-default-large, active)
 - [[kingjones-strix-mtp]] — the RETIRED kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (superseded 2026-08-25)
+- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook for frank-default-large (active): serving, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
 - [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
 - [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
 - [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)