Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context

frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache, and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning is on with preserve_thinking + preserve_reasoning in the chat kwargs. - frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on, REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking, preserve_reasoning} - serve-frank-large.sh: standalone draft-mtp profile (already switched off the Kairic/PromptForge path) - download-q8agent.sh: reproducible HF fetch of the three payload files - docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook - AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note - qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB (applied 2026-08-25), full 262k KV + Q8 payload fit in one slot - frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.
docs: add SearXNG providers guide for unblocking Google behind Gluetun VPN
2026-08-25 07:12:02 -07:00 · 2026-08-25 06:22:57 -07:00 · 2026-08-25 04:41:04 -07:00 · 2026-08-25 04:20:15 -07:00 · 2026-08-25 04:13:00 -07:00 · 2026-08-25 04:09:12 -07:00
17 changed files with 1248 additions and 211 deletions
@@ -27,14 +27,17 @@ host.
 | Target | Runtime | Service | Model | Port |
 |--------|---------|---------|-------|------|
 | Dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
-| Dankclaw | CIRU ROCmFPX llama.cpp, PromptForge v3 runtime (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3 (Q8-FFNEQ-LateQ6, full-W8 views, Output-K8 MTP proxy, BF16 vision), 262K context / 1 slot | 7770 |
+| Dankclaw | charlie12345/ROCmFPX llama.cpp | frank-default-large | kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT Q8_0_ROCMFPX (standalone MTP draft head, Turbo4 V cache, 262K context / 1 slot), reasoning on | 7770 |
 History: the 2026-08-17 redo retired the pipeline-parallel split and the
 FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
 Dankclaw. On 2026-08-21 a DFlash2 Vulkan deployment briefly replaced CIRU,
 then was reverted the same day for speed. On 2026-08-22 the service briefly
 swapped to the Kairic Edge IU4 runtime, then was reverted to v3 the same day
-(operator decision). Runbooks: [docs/LLM/index.md](docs/LLM/index.md).
+(operator decision). On 2026-08-25 the service switched to the kingjones
 Qwen3.8-27B-ROCmFPX-Q8_0-AGENT model (standalone MTP draft, 262144 context,
 reasoning on); the previous Kairic Edge model was retired the same day. Runbooks:
 [docs/LLM/index.md](docs/LLM/index.md).
 Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`.
 ## Repository Layout
@@ -179,17 +182,36 @@ When you rename a service or move a model to a new host, update
 ## Runtime & Troubleshooting
-### v3 CIRU ActiveFPX PromptForge llama.cpp (Dankclaw, ACTIVE)
+### kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT llama.cpp (Dankclaw, ACTIVE)
 The large service serves `kingjones/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT`
 (public) from `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/`
 with a charlie12345/ROCmFPX `llama-server` (build `build-kairic`). It runs a
 standalone MTP draft head on ROCm0, f16 K + TurboQuant V KV cache, a full
 262144 context / 1 slot, and reasoning on (Sharp template, budget -1,
 `--reasoning-format none`; chat kwargs keep `preserve_thinking` +
 `preserve_reasoning`). The payload (~28 GB main + 1.7 GB draft + 0.9 GB
 mmproj, ~30 GB total) plus the 262k KV cache fit in the lifted 114 GiB TTM/GTT
 budget (raised 2026-08-25). Env `scripts/frank-default-large.env`, serve
 `scripts/serve-frank-large.sh`, runbook
 [docs/LLM/kingjones-q8agent-rocmfpx.md](docs/LLM/kingjones-q8agent-rocmfpx.md).
 ### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, prior v3, 2026-08-18 to 2026-08-25)
 The v4 runtime serves `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (`eemin` namespace,
 public) from `/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server`
 (charlie12345/ROCmFPX commit `c49ebdbd` built with `-DGGML_HIP_ROCMI4_W4A4=ON`
 against system ROCm 7.2.4 / amdclang 22). The model is `Q4_0_ROCMI4` with an
 embedded NextN/MTP head (no separate draft file); the gfx1151 native IU4 WMMA
 instruction batch-verifies proposals. Profile: strict Qwen MTP
 (`--spec-mtp-strict-qwen`, n_max 16, p_min 0.60, backend sampling, all layers
 on ROCm0), `-t 16 -tb 32`, batch 512 / ubatch 256 (ubatch swappable via the
 ubatch bench on 8k context), f16 KV, full 262144 context / 1 slot, TEMPERATURE
 0.7 / TOP_P 0.95, reasoning on (deepseek, budget -1), `--no-mmap`. Two-file
 payload (~14.4 GiB) in `/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP/`. Full
 runbook:
 [docs/LLM/eemin-rocmi4-mtp.md](docs/LLM/eemin-rocmi4-mtp.md).
 The v3 runtime serves `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3`
 from `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`
 (PromptForge v2.3 + v3 Output-K8 runtime patch). The v3 profile:
 `PROMPTFORGE_MODE=m2048_fused_tail1476`, full-W8 companion views, Output-K8
 MTP draft proxy, BF16 mmproj vision, TEMPERATURE 0.7 / TOP_P 0.95,
 batch 4096 / ubatch 4096, 16 context checkpoints, reasoning on (deepseek,
 budget -1), `--no-mmap`. Five-file payload (~36.7 GiB) in
 `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. Full runbook:
 [docs/LLM/ciru-activefpx-promptforge.md](docs/LLM/ciru-activefpx-promptforge.md).
 ### Kairic Edge llama.cpp (Dankclaw, RETIRED experiment 2026-08-22)
@@ -295,9 +317,9 @@ graph TD
 - Never run `pkill -f` with a pattern that appears in your own command line.
  It kills your ssh session. Use `pkill -x <exact-name>`.
 - Model load is slow: Dankserver loads the 30.6 GB Q6_K_P through RAM
-  (`--no-mmap`, MoE experts on CPU) in a few minutes cold. Dankclaw v3
+  (`--no-mmap`, MoE experts on CPU) in a few minutes cold. Dankclaw loads the
-  loads the ~36.7 GiB five-file payload in ~18 s with a warm page cache,
+  ~30 GiB Q8_0 AGENT payload (main + MTP draft + mmproj) a few minutes cold
-  a few minutes cold. Health returns 503 while loading. Do not kill the
+  with a fresh KV cache; ~18 s with a warm page cache. Health returns 503 while loading. Do not kill the
  process. `deploy.sh` waits up to 20 minutes on both targets (see
  `LLM_WAIT`).
 - A client abort and a server release look different in the journal. A
@@ -0,0 +1,330 @@
 # SearXNG Providers
 How Google and other upstream search engines get blocked behind the Gluetun
 VPN tunnel, and how to unblock them in SearXNG.
 **Host:** Dankclaw — **Docs:** https://docs.searxng.org
 SearXNG aggregates ~50 upstream engines (Google, Bing, DuckDuckGo, Brave,
 Wikipedia, …). When all of them are queried through **one** VPN exit IP — as on
 Dankclaw, where SearXNG reuses Gluetun's tunnel — the rate-sensitive engines
 start answering with reCAPTCHA or HTTP 429. SearXNG then *suspends* the offending
 engine for an hour to 15 days and silently drops it from results. That is the
 "Google (and sometimes a few others) went unavailable" symptom.
 This page is the fixbook. Read [[SearXNG]] for the stack; the sub-pages below
 are the detail if you need it.
 ## Root cause (one line)
 All SearXNG engine traffic shares **one ProtonVPN exit IP** with no rotation, so
 Google/DuckDuckGo see one datacenter IP spamming ~50 paginated engines and
 rate-limit it → CAPTCHA → engine **suspension** → empty results.
 ## The suspension model (what "unavailable" actually is)
 An engine that returns 429 / reCAPTCHA / access-denied raises a named
 exception and is *suspended* (removed from result aggregation for a set time).
 Suspended engines are the usual cause of "gone":
 | Exception | Default suspension | Trigger |
 |-----------|--------------------|---------|
 | `SearxEngineTooManyRequests` | ~1 hour | HTTP 429 |
 | `SearxEngineAccessDenied` | ~1 day | 403 / access denied |
 | `SearxEngineCaptcha` | ~1 day | Google `/sorry/` or reCAPTCHA |
 | `recaptcha_SearxEngineCaptcha` | ~7 days | Google reCAPTCHA |
 | `cf_SearxEngineCaptcha` | ~15 days | Cloudflare edge CAPTCHA |
 | ordinary errors (timeout/parse) | 5–120 s | `ban_time_on_fail`/`max_ban_time_on_fail` |
 The values are `search.suspended_times` (+ `ban_time_on_fail`) in
 `settings.yml`. An engine that returns **0 results without an error** is *not*
 suspended — that is the separate "selector drift" problem (below).
 ## Two failure modes
 | Mode | Symptom | Root cause | Fix family |
 |------|---------|-----------|-----------|
 | **CAPTCHA / 429 / access-denied** | engine flips `unresponsive`, then `suspended` for hours | blocked from the shared VPN exit IP | routing/proxies + request tuning (see below) |
 | **Selector drift** | engine returns 0 results, no error banner | Google renamed its minified CSS classes the engine's hardcoded XPath looks for | engine config (duplicate engine / JSON feed) |
 Most "Google is broken on a VPN host" cases are a **mix of both**.
 ## Fix A — Engine config (settings.yml `engines:`)
 Important: SearXNG's engines are configured in the **`engines:` list inside
 `settings.yml`**, not a separate `engines.yml` file. With
 `use_default_settings: true` (our setting) your `engines:` list is *merged* over
 the built-in engines by `name` — you do **not** set `use_default_settings: false`.
 To add, override, disable, or prune:
 ```yaml
 # shared/searxng/settings.yml  (appended to the existing file)
 use_default_settings: true
 engines:
  # prune a broken engine entirely
  remove:
    - # (example) wikipedia   # drop an engine that never recovers
  # override + duplicate the Google engines so one survives the other's suspension
  - name: google
    weight: 3
    disabled: false
    timeout: 5.0                 # wait longer over the tunnel (global default is 2.0s)
    retry_on_http_error: [403, 429]
    display_error_messages: true
  - name: google2                # fresh engine, same module, survives google's CAPTCHA ban
    engine: google
    categories: general
    weight: 2
    timeout: 5.0
    disabled: false
 ```
 Key per-engine keys (each overwrites the engine-module default; see
 `settings_engines.html`):
 | Key | Effect |
 |-----|--------|
 | `name` | merge key (lowercase, no underscore) |
 | `engine` | Python module (`google`, `json_engine`, `xpath`) |
 | `categories` | categories it serves (also a comma string) |
 | `timeout` | per-engine HTTP timeout, overrides `outgoing.request_timeout` |
 | `weight` | ranks results higher |
 | `disabled` | off by default, selectable in /preferences |
 | `inactive` | `disabled` **and** removed from the UI — use to fully drop a broken engine |
 | `retry_on_http_error` | retry on status, e.g. `true`, `403`, `[403, 429]` |
 | `display_error_messages` | show/hide the engine's error banner |
 | `proxies` | per-engine egress override (see Fix C) |
 | `language` | force region, e.g. `en` |
 **Best duplicate trick:** re-declare `google` under a new `name` in a
 *different* category. When the primary `google` gets CAPTCHA'd it suspends, but
 `google2` still returns results, and vice-versa.
 **Parser caveat (read before fiddling with `parsers`):** the modern `google`,
 `duckduckgo`, and `brave` engines ship a **custom `response()`** with hardcoded
 XPath and *ignore* the generic `parsers`/`p_parsers` table. So `parsers:
 [scrapy_html]` under `name: google` mostly does nothing. To make Google
 robust against selector drift, either update the upstream `google.py`
 selectors (upgrade SearXNG) or run a **`json_engine`** against a JSON feed.
 Verify `parsers` consumption on the pinned version before depending on it.
 **JSON-feed fallback:** decouple parsing from Google's HTML churn.
 ```yaml
 engines:
  - name: google-json
    engine: json_engine
    categories: general
    paging: true
    search_url: 'https://www.googleapis.com/customsearch/v1?key={api_key}&q={query}&cx={cx}'
    results_query: results
    title_query: title
    url_query: link
    content_query: snippet
 ```
 ## Fix B — Request tuning (settings.yml `outgoing:`)
 SearXNG already rotates a **random browser User-Agent per outgoing request**
 (`gen_useragent()` in `online.py`) — there is **no native `useragents:` mapping
 key**. The blogs that point at a `useragents:` table describe the old
 `asciimoo/searx`; current SearXNG rotates automatically. Keep relying on it.
 Tune the `outgoing` block and the suspend model to survive load:
 ```yaml
 outgoing:
  request_timeout: 6.0             # raise from 2.0 so slow engines aren't dropped as unresponsive
  max_request_timeout: 15.0
  pool_connections: 200            # bigger pool = more concurrent connections per engine host
  pool_maxsize: 40                 # keep-alive connections per host
  enable_http2: true               # multiplexing = fewer distinct "new connection" events per engine
  retries: 3                       # on 429/5xx, retry against a different proxy + source IP
  useragent_suffix: "ops@yourhomelab.example"   # so a per-engine blocker can reach you
 search:
  ban_time_on_fail: 10
  max_ban_time_on_fail: 180
  suspended_times:
    SearxEngineTooManyRequests: 300     # 429 → 5 min before it retries (tune down to recover faster)
    SearxEngineCaptcha: 7200            # CAPTCHA → 2h before it retries
 ```
 Notes:
 - **Auto-suspend is the real "rate limit".** The old per-engine `rate_limit` /
  `time_between_requests` sleep field does **not** exist in current SearXNG —
  that is a historical-searx concept. The functional equivalent is the suspend
  schedule above (widen to back off before blocking, narrow to recover faster).
 - `request_timeout` interacts with the suspend model: too tight → engines look
  "unresponsive"; too loose → slow pages. Aim per-engine `timeout` at
  *proxy-add-on time + engine latency*, capped by `max_request_timeout`.
 - `safe_search: 0` when diagnosing — a too-strict global safe search can make an
  engine return zero rows and *look* blocked.
 ## Fix C — Routing & proxies (change which IP the upstreams exit as)
 Two independent levers: SearXNG's own upstream `proxies`, and the Gluetun VPN
 strategy. The decisive fact for our topology: a proxy that runs **inside the
 same Gluetun netns** exits from the *same* single IP, so round-robin among
 same-IP proxies **does not** unblock. The proxy must egress from a **different,
 clean IP**.
 ```mermaid
 graph LR
    SE[SearXNG] -->|default| GW[Gluetun #1 netns] -->|exit| IP1[ProtonVPN IP A — gets blocked]
    SE -->|per-engine proxies| GW2[Gluetun #2 / dedicated IP] -->|exit| IP2[ProtonVPN IP B — clean]
 ```
 **Option 1 — per-engine `proxies` (cheapest, settings.yml only).** Point the
 heaviest/blocked engines (Google and Google-hosted) at a proxy that exits from a
 clean IP:
 ```yaml
 engines:
  - name: google
    proxies:
      http:
        - http://gluetun-upstream:8080
      https:
        - http://gluetun-upstream:8080
        - socks5h://gluetun-upstream:8388
  - name: google news
    engine: google_news
    proxies:
      https:
        - http://gluetun-upstream:8080
 ```
 `using_tor_proxy` follows the same per-engine/global pattern.
 **Option 2 — dedicated second Gluetun for SearXNG's upstreams.** Run a second
 Gluetun on a different ProtonVPN server/geo, publish its built-in HTTP proxy
 (`8080`, `HTTPPROXY=on`), and point the Google engines at it. SearXNG inbound
 (still on the main host/Caddy) stays put; only *upstream* requests re-route.
 Gives a second isolated exit IP and reputation pool.
 - **Cons:** a second ProtenVPN device slot, a new container, port publishing
  moves to Gluetun, and **split-brain DNS** (below).
 **Option 3 — dedicated ProtonVPN IP.** A static, exclusive exit IP that other
 subscribers can't pollute. Requires a **ProtonVPN Business plan** as of the
 sources reviewed (the long-requested *personal*-plan dedicated IP is still only
 on uservoice — flag: verify your plan). Wire it in Gluetun by pinning the node
 via `SERVER_HOSTNAMES`.
 **Option 4 — rotate the exit.** List `SERVER_COUNTRIES`/`SERVER_CITIES`/
 `SERVER_HOSTNAMES` so Gluetun picks from several physical exit servers. Caveat:
 rotating the **shared** Gluetun rotates the exit for *all* attached
 containers. To rotate only SearXNG's upstreams, use Option 1 → a dedicated
 gateway.
 > ⚠️ **Split-brain DNS:** a container sharing Gluetun's netns
 > (`Network=gluetun.container`) loses Docker's internal service-name DNS — it
 > can't resolve siblings by name. Use a static IP or `extra_hosts`/`~/.config/
 > containers/systemd/<unit>.container` `Environment` to inject names into the
 > tunnel's `/etc/hosts`.
 ## Fix D — Verify an engine is unblocked
 Three layers, cheapest first. All queries should hit the path production uses
 (the **published Gluetun port → ProtonVPN exit**, not a direct host port).
 **1. Isolate one engine with `/json`.** Enable `json` in `search.formats`
 (otherwise `format=json` → 403), then query a single engine:
 ```bash
 BASE="http://192.168.0.143:8081"   # published port (ProtonVPN exit)
 curl -s "$BASE/search?q=test&engines=google&format=json" \
  | jq -r '.unresponsive_engines[]? | select(.engine=="google")'
 ```
 Inspect:
 - `content[].engine` — which engine produced each result.
 - `unresponsive_engines[]` — `{engine, error_type, suspended}`. `error_type`
  tells you *why* (`SearxEngineTooManyRequests`, `SearxEngineCaptcha`,
  `timeout`, `SearxEngineAccessDenied`); `suspended: true` = auto-banned right
  now.
 - `timings[].engine` — per-engine latency.
 Full status check (bash + jq):
 ```bash
 BASE="http://192.168.0.143:8081"
 for e in google bing duckduckgo brave startpage; do
  curl -s "$BASE/search?q=test&engines=$e&format=json" \
    | jq -r --arg e "$e" '
        (.unresponsive_engines // [])
        | map(select(.engine==$e))
        | if length==0 then "\($e): OK"
          else .[] | "\($e): UNRESPONSIVE (\(.error_type), suspended=\(.suspended))" end'
 done
 ```
 **2. `/config` REST endpoint** lists every engine + enabled/disabled +
 categories — an inventory of what is live. `?engines=<name>` isolates a single
 provider; `/debug` adds engine diagnostics.
 **3. `check.searx.space`** — the hosted official checker probes every engine of
 your instance with a **headless Firefox** (catches Cloudflare/CAPTCHA blocks a
 plain HTTP probe misses) and posts results on searx.space. Add your instance via
 a GitHub issue in `searxng/searx-instances`, then allow `check.searx.space` IPs
 to reach the SearXNG port.
 Local self-test CLI (`kvch/searx-checker`):
 ```bash
 python3 checker/checker.py http://192.168.0.143:8081
 # google.OK  bing.OK  gitlab......ERROR
 ```
 Verify through the **published Gluetun port** (realistic ProtonVPN-exit block
 signal), not a direct host port (which uses the host LAN IP and can be
 misleading).
 ## Recommended path for Dankclaw (ranked)
 1. **Verify first** — `/json` + `?engines=google` and the searx-checker to see
   exactly which engines are down and why. Confirm the running SearXNG version
   (check whether `parsers`/`rate_limit` exist in the pinned build).
 2. **Tighten Fix B** — raise `outgoing.request_timeout`, add `retry_on_http_error:
   [403, 429]` on the Google engines, widen `suspended_times`. Cheapest, no new
   infra.
 3. **Duplicate the Google engines** (Fix A) so a CAPTCHA ban on one leaves the
   other returning results.
 4. **If Google still re-blocks** — Fix C Option 1: point the Google engines at a
   dedicated-IP proxy (Option 2 dedicated second Gluetun if reputation
   isolation is needed). Avoid the same-netns-proxy trap.
 ---
 ## Sources
 - [SearXNG settings: engines](https://docs.searxng.org/admin/settings/settings_engines.html) — per-engine keys.
 - [SearXNG settings: outgoing](https://docs.searxng.org/admin/settings/settings_outgoing.html) — `request_timeout`, pool, `retries`, `proxies`, `source_ips`.
 - [SearXNG settings: search](https://docs.searxng.org/admin/settings/settings_search.html) — `suspended_times`, `ban_time_on_fail`.
 - [SearXNG settings (merge + remove/keep_only)](https://docs.searxng.org/admin/settings/settings.html) — `use_default_settings` semantics.
 - [Google engine docs](https://docs.searxng.org/dev/engines/online/google.html) — `detect_google_sorry`, WML/Nokia XML, `max_page`.
 - [json_engine](https://docs.searxng.org/dev/engines/json_engine.html) — `json` parser config.
 - [SearXNG exceptions](https://docs.searxng.org/src/searx.exceptions.html) — exception types and suspend settings.
 - [Google engine stops yielding results #5286](https://github.com/searxng/searxng/blob/master/searx/engines/google.py) — reproducible "Google unavailable".
 - [DuckDuckGo CAPTCHA #4824](https://github.com/searxng/searxng/issues/4824) — per-engine CAPTCHA signature.
 - [Google engine actively blocking IPs #2515](https://github.com/searxng/searxng/issues/2515)
 - [answer-captcha from server's IP](https://docs.searxng.org/admin/answer-captcha.html) — block is tied to the egress IP.
 - [SearXNG rate limits / 429 (SSD Nodes)](https://www.ssdnodes.com/learn/searxng-rate-limits-and-429-errors) — suspension table, `/stats/errors`.
 - [How search engines detect scrapers (Crawlbase)](https://crawlbase.com/blog/how-search-engines-detect-scrapers/) — IP reputation + rate + CAPTCHA mechanics.
 - [check.searx.space](https://github.com/searxng/searx-space) + [kvch/searx-checker](https://github.com/kvch/searx-checker) — the checkers.
 - [searx-instances — how to be checked](https://github.com/searxng/searx-instances) — add your instance + allow checker IPs.
 - [ProtonVPN — Dedicated IP addresses](https://proton.me/support/dedicated-ips) + [Benefits](https://proton.me/support/dedicated-ips) — IP-reputation reasoning; **Business plan** (flag: personal dedicated IP still on uservoice).
 - [Gluetun wiki — ProtonVPN provider](https://github.com/qdm12/gluetun-wiki/blob/main/setup/providers/protonvpn.md) — `SERVER_HOSTNAMES` narrowest-filter risk.
 - [Gluetun wiki — Built-in Proxies](https://deepwiki.com/qdm12/gluetun-wiki/4.5-built-in-proxies) — HTTP `8080` + Shadowsocks `8388`.
 - [Gluetun wiki — Container networking](https://deepwiki.com/qdm12/gluetun-wiki/6.5-container-networking) — netns sharing, port publishing on Gluetun.
 - [Gluetun inter-network DNS failures (Jacob Jangles)](https://jacobjangles.com/posts/fixing-gluetuns-inter-network-dns-failures-in-docker/) — split-brain DNS + fix.
 ## See Also
 - [[SearXNG]] — the stack, routing, limiter, health
 - [[Gluetun]] — the VPN tunnel (where the exit IP comes from)
 - [[Dankclaw Stack]] — the service the SearXNG stack sits in
 - [[Service Research Brief]] — the research that produced these pages
@@ -94,7 +94,22 @@ The rate limiter's JSON cap is `API_MAX = 4` requests per IP per hour. The const
 Reach SearXNG through Caddy (`search.dankclaw.local`) for browser use. A local subagent can hit the published port directly because its slirp source is now trusted. The searxng subnet 10.89.2.0/24 is in `trusted_proxies`, so Caddy's `X-Forwarded-For` is honored. The subnet is pinned in `searxng.network`, so it stays stable across reboots.
-## Gotchas
+## Unblocking Google & Search Providers
 SearXNG queries all ~50 upstream engines from one Gluetun VPN exit IP. The
 rate-sensitive engines (Google, DuckDuckGo) then answer with reCAPTCHA / 429,
 and SearXNG **suspends** the offending engine for hours to days, which is what
 shows up as "Google went unavailable". This is upstream, not local.
 Fixes (cheapest first): tune the `outgoing` block (`request_timeout`,
 `retry_on_http_error`, `suspended_times`), duplicate the `google` engine under a
 fresh `name` so a CAPTCHA ban on one leaves the other returning results, and —
 if Google still re-blocks — route the Google engines at a clean-IP proxy via
 per-engine `proxies` in `settings.yml`. See [[SearXNG Providers]] for the full
 root cause, the suspend-model table, the `engines:` config, and the
 `/json` verification recipe.
 ### Gotchas
 - The JSON API rate cap (4 per 3600s per IP) is hardcoded in Python, not settable in `limiter.toml`. The only bypass is `pass_ip`. Local subagents are covered because their slirp/loopback/LAN sources are in `pass_ip`.
 - The limiter stays armed for untrusted sources (a spoofed public IP still 429s). Do not disable it. Widen only `pass_ip` for sources you trust on this host.
@@ -106,4 +121,5 @@ Reach SearXNG through Caddy (`search.dankclaw.local`) for browser use. A local s
 - [[Dankclaw Stack]]
 - [[Gluetun]] — the VPN container
- Knowledge base: [[Gluetun]], [[SearXNG]], [[Hermes]]
+- [[SearXNG Providers]] — unblocking Google & other upstream engines (VPN exit IP, engine config, proxies, verification)
 - Knowledge base: [[Gluetun]], [[SearXNG]], [[SearXNG Providers]], [[Hermes]]
@@ -21,20 +21,24 @@ marked "verify first". Re-verified 2026-08-15/16; corrections marked below.
 - **KV cache**: production runs `q8_0` K **and** `q8_0` V (both quantized).
 - **Spec**: DFlash/DSpark are retired. Production is `--spec-default`
  (ngram-mod) with draft-mtp first. See [[qwen3.8-spec-matrix]].
- **TTM**: still 15.24 GiB as of 2026-08-16 — the modprobe fix is still
+- **TTM**: raised to 114 GiB on 2026-08-25, set live (no reboot) and
-  pending a reboot.
+  persisted in `/etc/modprobe.d/ttm.conf`. The effective VRAM ceiling that
  once held the service at ~15 GiB is gone.
 ## Verified Memory Picture (measured 2026-08-11)
 | Figure | Value | Source |
 | ------ | ----- | ------ |
 | OS physical RAM | 30.5 GiB | `/proc/meminfo` |
-| TTM pages_limit | 15.24 GiB (3,995,281 x 4KiB) | `/sys/module/ttm/parameters/pages_limit` |
+| TTM pages_limit | 114 GiB (29,883,536 x 4KiB) — raised 2026-08-25 | `/sys/module/ttm/parameters/pages_limit` |
 | ROCm VRAM total | 96.0 GiB | `rocm-smi --showmeminfo vram` |
 | ROCm VRAM used | 76.2 GiB | `rocm-smi` while serving 3x200700 |
-The OS accounts only ~30.5 GiB physical RAM, and the TTM GPU-addressable pool
+The OS accounts only ~30.5 GiB physical RAM. The TTM GPU-addressable pool
-is capped at ~15.24 GiB. ROCm/SMI still reports a 96 GiB VRAM carve-out (VGM).
+was capped at ~15.24 GiB (the real pre-2026-08-25 ceiling); that cap is now
 lifted to 114 GiB. ROCm/SMI still reports a 96 GiB VRAM carve-out (VGM) — a
 different, driver-reported figure that has always been larger than the kernel
 cap.
 This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
 limits documented in [[frank-default-large-benchmark]].
@@ -77,7 +81,7 @@ All require root and a reboot for the modprobe change.
 | Setting | Action | Effect | Priority |
 | ------- | ------ | ------ | -------- |
-| TTM GPU pool | Set `amd-ttm --set 96` (write `/etc/modprobe.d/ttm.conf`), reboot | Lifts the 15 GiB TTM cap toward full RAM; more headroom for GPU buffers | High |
+| TTM GPU pool | `/etc/modprobe.d/ttm.conf` → `options ttm pages_limit=29883536 page_pool_size=29883536` (applied 2026-08-25, set live then persisted) | Lifts the 15 GiB cap to 114 GiB; the full 262k KV + Q8 AGENT payload fit in one slot | Done |
 | Power profile | **N/A on this machine** (2026-08-15): no `power_profile` sysfs; GPU DPM already `performance` | none | ~~Medium~~ |
 | NVMe scheduler | Already `none` (`[none]` confirmed) | No change | Done |
 | BIOS VGM | Set Variable Graphics Memory / UMA to "High" in the EVO-X2 BIOS | Max dedicated VRAM for the GPU carve-out; verify in the live BIOS | Verify |
@@ -18,7 +18,7 @@ repository to a running service.
 | Target | Host | Runtime | Service | Port |
 |--------|------|---------|---------|------|
 | Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q6_K, native MTP draft, selected MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
-| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (Qwen3.8-27B-Uncensored-ROCmFP4 Q6 AGENT, standalone MTP Q4_0 head, 262K ctx / 1 slot) | frank-default-large | 7770 |
+| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (caf-eemin Qwen3.8-27B ROCmI4 Q4_0, embedded MTP, ROCmI4 W4A4, 262K ctx / 1 slot) | frank-default-large | 7770 |
 ## Workflow
@@ -137,24 +137,22 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
 ### Dankclaw — frank-default-large
-The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
+The active service on Dankclaw uses the ROCmFPX llama.cpp runtime to serve
-`kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` (Q6_0 AGENT, dense)
+[kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF)
-under the alias `frank-default-large` on port 7770. It runs
+under the alias `frank-default-large` on port 7770. The main model is
-`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
+`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`; the standalone MTP draft is
-7.15 SDK in the retired ling3-ciru venv) with a standalone MTP Q4_0 draft
+`mtp-Qwen3.8-27B-Q4_0.gguf`, and the BF16 vision projector is loaded alongside
-head (`--model-draft` and `--spec-type draft-mtp`), f16 KV, full 262144
+it. The K-side KV cache remains `f16`; the V-side cache uses `turbo4`. The
-context (1 slot. Concurrent requests queue; the server never preempts a
+service keeps the existing 65536 context / 1-slot profile. The unit file is
-running generation), batch 4096 / ubatch 4096, 16 context checkpoints,
+`scripts/frank-default-large.service`; it launches through
 8 GiB prompt cache, and reasoning in deepseek format. The unit file is
 `scripts/frank-default-large.service`. It launches through
 `scripts/serve-frank-large.sh`, which reads tunables from
 `scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
 See [[kingjones-strix-mtp]].
-> The kingjones Q6 AGENT model has served the slot since 2026-08-24; the
+> The kingjones ROCmFP4 STRIX MTP model served the slot from 2026-08-24; it
-> v3 PromptForge payload is retired on this slot.
+> was replaced by the caf-eemin ROCmI4 embedded-MTP model on 2026-08-25. The
-> Restored 2026-08-21. The intermediate DFlash2 Vulkan deployment was
+> v3 PromptForge payload and the CIRU ROCmFPX runtime are retired on this slot.
-> retired the same day. See [[dflash2-large-vulkan]] for the archived runbook.
+> The intermediate DFlash2 Vulkan deployment was retired the same day (2026-08-21).
 > See [[dflash2-large-vulkan]] for the archived runbook.
 ## Validation
@@ -0,0 +1,184 @@
 ---
 type: reference
 title: caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw
 description: caf-eemin/Qwen3.8-27B-ROCmI4-MTP-GGUF (Q4_0_ROCMI4, embedded NextN/MTP head) served by the charlie12345/ROCmFPX llama.cpp runtime (ROCmI4 W4A4) on the Strix Halo iGPU. frank-default-large.
 tags:
  - llama.cpp
  - rocm
  - dankclaw
  - gfx1151
  - rocmi4
  - mtp
  - iu4
 timestamp: 2026-08-25
 ---
 # caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw
 > **Active 2026-08-25.** `frank-default-large` serves this model. It replaced
 > the kingjones777 ROCmFP4 STRIX MTP payload on the same port and the same
 > Strix Halo iGPU, running the new charlie12345/ROCmFPX ROCmI4 W4A4 build. See
 > [[kingjones-strix-mtp]] for the retired page.
 `frank-default-large` (port 7770) serves
 `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (published under the public
 `eemin` namespace) from the **charlie12345/ROCmFPX** llama.cpp fork built with
 the opt-in `ROCmI4 W4A4` path. The model is `Q4_0_ROCMI4`: signed four-bit
 weights packed two per byte, quantized from the BF16 Qwen3.8-27B and carrying
 its **embedded NextN/MTP head** in the main GGUF — so there is no separate
 draft file. The native gfx1151 `v_wmma_i32_16x16x16_iu4` matrix instruction
 batch-verifies the MTP proposals for higher decode throughput.
 ## What This Is
 **ROCmI4** is the GGUF weight format (signed 4-bit codes). **IU4** is the AMD
 WMMA instruction spelling that consumes them. **W4A4** means four-bit weights
 and four-bit activations during the accelerated matrix multiply. Ordinary
 one-token decode stays memory-bandwidth-bound (MMVQ); batched prompt processing
 and batched MTP target verification use the IU4 MMQ path. W4A4 is a lossy
 activation-quant speed path — treat it as a speed/quality tradeoff.
 The model is a conversion/quantization of `Qwen/Qwen3.8-27B` (Unsloth lineage,
 Apache-2.0), not a retrained model. It reports architecture `qwen35`
 (M-RoPE), ~27.32B params, native 262144 context, 866 tensors, ftype 108.
 ## Model Files
 | File | Size | SHA-256 |
 |------|------|---------|
 | `Qwen3.8-27B-Q4_0_ROCMI4.gguf` | 13.50 GiB (14,534,384,384 B) | `ad51ecbee55930ababc629c377943a7bbe50f8aafcf32340c7cdae6fa6b7bb3c` |
 | `mmproj-BF16.gguf` (vision, optional) | 0.87 GiB (931,146,432 B) | `83ee4f4f205fa514161778c41df1ea14144faa0f713510893b63c2395f5c2d53` |
 Store under `/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP/`. The vision
 projector is optional for text-only use.
 ## Runtime Build
 The stock llama.cpp build does not understand `Q4_0_ROCMI4` or the qualified
 IU4/MTP path. Use charlie12345/ROCmFPX at the pinned commit, built HIP-only
 (so the combined HIP/Vulkan scheduler cannot route the ROCmI4 tensors to
 Vulkan, which has no shader for this type) with the W4A4 option on.
 | Artifact | Path |
 |----------|------|
 | Fork + pinned commit | `/home/dank/rocmi4-qwen38/ROCmFPX` @ `c49ebdbd5c9f01ec242369f9e7f7967855f80cba` |
 | Build | `cmake -S . -B build-rocmi4-w4a4 -G Ninja -DGGML_HIP=ON -DGGML_VULKAN=OFF -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DGGML_HIP_ROCMI4_W4A4=ON -DLLAMA_BUILD_SERVER=ON -DLLAMA_BUILD_WEBUI=OFF` |
 | Binary | `/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server` (version 257, `c49ebdb`) |
 | ROCm | system `/opt/rocm-7.2.4`, amdclang 22 |
 Build a separate directory so the exact-int8 path (the default branch) stays
 available. Rollback: disable `-DGGML_HIP_ROCMI4_W4A4=ON` and rebuild — no model
 conversion or data change needed. The build prints
 `ROCmI4 W4A4: enabled for device 0` when the accelerated path is active.
 ## Model Download
 Downloaded from the public
 `eemin/Qwen3.8-27B-ROCmI4-MTP-GGUF` repo (the card's download section still
 names the still-private `cafonz/…`; use `eemin`). Verify against the repo's
 `checksums.sha256` after download:
 ```
 cd /home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP
 sha256sum -c checksums.sha256
 ```
 ## Launch Profile
 The qualified MTP profile (from the model card and PR #94, verified on the
 live service): strict Qwen MTP, backend sampling, `n_max 4`, `p_min 0.60`.
 ```mermaid
 graph TD
    A[llama-server charlie12345 ROCmFPX c49ebdb<br/>ROCmI4 W4A4 enabled] --> B[load Q4_0_ROCMI4 GGUF + mmproj<br/>-ngl 999 -dev ROCm0]
    B --> C[draft-mtp on ROCm0<br/>--spec-draft-ngl all]
    C --> D[strict Qwen MTP<br/>n_max 4 / p_min 0.60<br/>backend sampling]
    D --> E[Sharp chat template<br/>reasoning deepseek]
 ```
 Key flags (see `scripts/serve-frank-large.sh`):
 ```
 --spec-type draft-mtp --spec-mtp-strict-qwen
 --spec-draft-device ROCm0 --spec-draft-ngl all
 --spec-draft-type-k f16 --spec-draft-type-v f16
 --spec-draft-n-max 4 --spec-draft-n-min 0
 --spec-draft-p-min 0.60 --spec-draft-backend-sampling
 ```
 Because the MTP head is **embedded**, no `--model-draft` is passed and the
 draft uses all layers on ROCm0. `--spec-mtp-strict-qwen` preserves the target
 path's greedy decisions; it does not remove the W4A4 activation approximation.
 ### Serving settings
 - 65536 context (192K below the native 262144 maximum), 1 slot, f16 KV,
  context checkpoints, prompt cache. The lower limit leaves headroom for the
  model plus KV cache in the host's unified-memory budget. Concurrent requests
  queue; the server never preempts a running generation.
 - `-t 16 -tb 32`, batch 512 / ubatch 256 (the qualified W4A4 profile).
 - `--temp 0.7 --top-p 0.95`, reasoning on (`deepseek` puts `<think>` into
  `message.reasoning_content`).
 - Qwen-Sharp chat template (`--chat-template-file`), `enable_thinking` +
  `preserve_thinking` on, preserving the peg-native + deepseek shape the Pi
  agent expects.
 - Strix Halo unified memory + native gfx1151 detection: `HSA_OVERRIDE_GFX_VERSION=11.5.1`,
  `GGML_HIP_ENABLE_UNIFIED_MEMORY=1`.
 ### Service
 - Unit: `frank-default-large.service`, port 7770
 - Script: `scripts/serve-frank-large.sh`, env `scripts/frank-default-large.env`
 ## Tuning MTP
 MTP gains are content-dependent: predictable code, structured data, and lists
 accept longer drafts than creative prose. The active conservative profile is
 `n4 / p0.60`; the previously qualified starting point was `n16 / p0.60`.
 ## ubatch Benchmark
 Decode throughput depends on the ubatch (per-eval token batch). Run the sweep
 on 8k contexts, 512 → 4096 in steps of 512, with a fixed batch:
 ```
 STOPSVC=1 ./scripts/bench-ubatch-rocmi4.sh
 ```
 The script stops `frank-default-large` for a clean GPU, boots one server
 instance per ubatch on a scratch port, reads authoritative `print_timing` tok/s,
 and restarts the service. See [[frank-default-large-benchmark]].
 ## Known Issues
 1. **gfx1151 only** for the W4A4 path. Other GPUs fall back to exact int8 MMQ.
 2. **Memory pressure:** the model is 13.5 GiB. Under `--no-mmap` with full
   offload it needs the whole tensor set at once; leave room in the 64 GiB
   unified pool (rocm-smi figure). Do not run a bench with the live service up
   or it OOMs at load.
 3. **`Q4_0_ROCMI4` + MTP flags need the ROCmFPX fork**; stock llama.cpp may
   reject the file or drop the optimized path.
 4. **W4A4 is lossy** (+~5.4% perplexity vs exact int8). Opt-in speed path.
 ## Measured Results (qualification)
 From the model card, gfx1151, pinned build, W4A4 + strict MTP-16:
 | Result | Value |
 |--------|-------|
 | HumanEval pilot, W4A4 IU4 (mean) | 49.40 tok/s |
 | Full HumanEval, W4A4 (mean / median) | 44.39 / 45.23 tok/s |
 | HumanEval / HumanEval+ pass@1 | 94.5% / 91.5% |
 | Non-speculative tg128 | ~13.8 tok/s |
 Local smoke test 2026-08-25 on the iGPU: model loaded, strict-MTP active,
 draft acceptance 0.74 (mean 7.67), generation ~25 tok/s at a small output.
 The ubatch sweep above gives the production ubatch choice.
 ## See Also
 - [[deployment]] — deploy and update process
 - [[kingjones-strix-mtp]] — the retired ROCmFP4 predecessor on this slot
 - [[ciru-activefpx-promptforge]] — the retired v3 CIRU PromptForge page
 - [[Dankclaw System Specs]] — the box
 - [[frank-default-large-benchmark]] — backend/spec benchmark history
@@ -1,10 +1,10 @@
 # LLM Fleet — Knowledge Base
 Documentation for the LLM inference fleet on two local machines
-(2026-08-24): upstream llama.cpp on Dankserver (frank-default-small,
+(2026-09-01): upstream llama.cpp on Dankserver (frank-default-small,
-Ornith-1.5-35B-A3B CRACK) and CIRU ROCmFPX llama.cpp on Dankclaw
+Ornith-1.5-35B-A3B CRACK) and ROCmFPX llama.cpp on Dankclaw
-(frank-default-large, kingjones Qwen3.8-27B ROCmFP4). Both services
+(frank-default-large, kingjones Qwen3.8-27B ROCmFPX Q8 AGENT). Both
-run the Qwen-Sharp chat template.
+services run the Qwen-Sharp chat template.
 > 2026-08-22: the former `vllmScripts` repository is merged into the
 > serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -28,8 +28,8 @@ ngram-mod, selected experts offloaded via override, 491520 ctx]
        D[gfx1151 iGPU 64 GiB unified memory]
        E[CIRU ROCmFPX llama.cpp PromptForge v3
 frank-default-large, port 7770]
-        F[Qwen3.8-27B-Uncensored-ROCmFP4 Q6_0 AGENT
+        F[Qwen3.8-27B ROCmFPX Q8_0 AGENT
-MTP draft + Sharp template, 262K ctx / 1 slot]
+MTP draft + Turbo4 V cache, 262K ctx / 1 slot]
        D --> E --> F
    end
@@ -48,8 +48,14 @@ MTP draft + Sharp template, 262K ctx / 1 slot]
 - [[deployment]] — deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model
+- [kingjones Qwen3.8-27B ROCmFPX Q8 AGENT](https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF) — active frank-default-large model on Dankclaw; standalone MTP and Turbo4 V cache
-  on Dankclaw (frank-default-large, active 2026-08-24)
+- [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook: serving profile, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
 - [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
  on Dankclaw (retired)
 - [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
  Dankserver (frank-default-small, active 2026-08-24)
 - [[kingjones-strix-mtp]] — the retired kingjones Qwen3.8-27B ROCmFP4 STRIX MTP
  model on Dankclaw (superseded 2026-08-25 by the ROCmI4 embedded-MTP model)
 - [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on
  Dankserver (frank-default-small, active 2026-08-24)
 - [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime
@@ -0,0 +1,165 @@
 # frank-default-large: kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
 **Status:** ACTIVE on Dankclaw (since 2026-08-25).
 **Alias:** `frank-default-large`, port **7770**.
 The large service serves the kingjones `Qwen3.8-27B-ROCmFPX-Q8_0-AGENT` GGUF
 from `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/` with a
 charlie12345/ROCmFPX `llama-server` (build `build-kairic`). It runs a
 **standalone MTP draft head** (loaded separately, on ROCm0), **TurboQuant** V
 KV cache, the **Sharp** chat template with **reasoning on**, and a **full
 262144 context / 1 slot** profile. The whole payload plus the 262k KV cache
 fit in the lifted **114 GiB TTM/GTT** budget (see the 114 GiB change in
 [[Dankclaw Improvements]] and [[qwen3.8-strix-optimizations]]).
 | Figure | Value |
 | ------ | ----- |
 | Model | kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT (`Q8_0_ROCMFPX`) |
 | HF repo | `kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF` (public) |
 | Runtime | charlie12345/ROCmFPX `llama-server`, `build-kairic` |
 | Draft head | standalone `mtp-Qwen3.8-27B-Q4_0.gguf` on ROCm0 |
 | KV cache | f16 K, TurboQuant (`turbo4`) V, full 262144 context, 1 slot |
 | Reasoning | on, `deepseek`, budget `-1`, `--reasoning-format none` |
 | Chat template | Sharp (repo-tracked), `enable_thinking` + `preserve_thinking` on |
 | Host path | `/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT/` |
 | Tunables | `scripts/frank-default-large.env` |
 | Serve script | `scripts/serve-frank-large.sh` |
 ## Model files
 Three files, from the HF repo above:
 | File | Size | Purpose |
 | ---- | ---- | ------- |
 | `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` | 28.21 GB | main weights |
 | `mtp-Qwen3.8-27B-Q4_0.gguf` | 1.68 GB | standalone MTP draft head (loaded by the script) |
 | `mmproj-Qwen3.8-27B-BF16.gguf` | 0.93 GB | vision projector (`--mmproj`) |
 The repo also has a `mtp-Qwen3.8-27B-Q8_0.gguf` (3.16 GB) draft; the env uses
 the Q4_0 draft. Download them from the resolve URLs below (they are large —
 use the token in `~/.cache/huggingface/token`).
 ## Download
 ```bash
 # on Dankclaw
 TOKEN=$(cat ~/.cache/huggingface/token)
 REPO=kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF
 DIR=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT
 mkdir -p "$DIR"; cd "$DIR"
 dl(){ curl -fsSL -C - -o "$1" --retry 5 --retry-delay 3 --retry-max-time 3600 \
  -H "Authorization: Bearer $TOKEN" \
  "https://huggingface.co/$REPO/resolve/main/$1"; echo "done: $1"; }
 dl Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf
 dl mmproj-Qwen3.8-27B-BF16.gguf
 dl mtp-Qwen3.8-27B-Q4_0.gguf
 ```
 The serve script checks each artifact is present and readable before launch
 (`scripts/frank-default-large.env` -> the four `MODEL_*` files plus the chat
 template). A missing file fails the artifact check with `missing or
 unreadable required artifact`.
 ## Serving profile
 `scripts/serve-frank-large.sh` builds the `llama-server` call from the env:
 ```
 /dev/llama-server -m <main> --alias frank-default-large
  --host 0.0.0.0 --port 7770 --jinja
  -dev ROCm0 -ngl 999 -np 1 -c 262144
  -b 512 -ub 512 -fa on
  -ctk f16 -ctv turbo4 -t 16 -tb 32
  -ctxcp 32 --cache-ram 8192 --cache-prompt --cache-idle-slots --metrics
  --spec-type draft-mtp --model-draft <mtp-Q4_0>
  --mmproj <mmproj-BF16>
  --spec-draft-device ROCm0 --spec-draft-ngl 999
  --spec-draft-type-k f16 --spec-draft-type-v turbo4
  --spec-draft-n-max 4 --spec-draft-n-min 0
  --spec-draft-p-min 0.0 --spec-draft-p-split 0.10
  --spec-draft-backend-sampling
  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0
  --presence-penalty 1.5
  --reasoning on --reasoning-format none --reasoning-budget -1
 ```
 Key choices:
 - **Standalone MTP draft** (`--model-draft`, `--spec-type draft-mtp`). The
  AGENT GGUF has no embedded draft head; the draft head is a separate file on
  ROCm0. This matches the Qwen3.8-27B AGENT shape (`--spec-draft-type-k
  f16 --spec-draft-type-v turbo4`).
 - **f16 K + TurboQuant V** cache. f16 K is the safe long-ctx default; TurboQuant
  halves the V cache at 262k while keeping quality (see the ubatch/threads-batch
  notes in [[qwen3.8-strix-optimizations]]).
 - **Full 262144 context, 1 slot.** The whole context KV cache plus the ~30 GB
  payload (main + draft + mmproj) fit in the 114 GiB budget. Concurrent
  requests queue; the server never preempts a running generation.
 - **Reasoning on.** The Sharp template drives thinking (medium effort);
  `--reasoning-format none` lets the template project the reasoning content into
  `reasoning_content`. `--reasoning-budget -1` = no budget cap.
 ## Reasoning + chat template
 The Sharp template (`scripts/chat-template.jinja`, repo-tracked) is a
 Qwen-AGENT shape. Env `CHAT_TEMPLATE_KWARGS` keeps the reasoning trail in the
 conversation:
 ```json
 {"enable_thinking": true, "preserve_thinking": true, "preserve_reasoning": true}
 ```
 `preserve_thinking: true` keeps the model's prior thinking in the next prompt
 (the user's "preserve thinking" request). `preserve_reasoning: true` keeps the
 model's prior reasoning trace across requests too (OpenWebUI sends
 `preserve_reasoning`/`preserve_thinking: true` when the conversation needs the
 full trail). For multi-turn reasoning, the template stays in the reasoning
 shape and `enable_thinking: true` per request.
 ## Memory: the 114 GiB TTM/GTT requirement
 The 262k f16 KV cache is large (~a few GB) and the payload is ~30 GB. Both live
 in the Strix Halo unified-memory GTT. The TTM `pages_limit` was raised to
 114 GiB on 2026-08-25 (written live to
 `/sys/module/ttm/parameters/pages_limit`, then persisted in
 `/etc/modprobe.d/ttm.conf`). Without the raise the effective ceiling is ~15 GiB
 and a 28 GB + 262k-KV load stalls. The GTT page-limit change is per the
 `qwen3.8-strix-optimizations` table item 5.
 ## Health + troubleshooting
 - Health: `curl http://192.168.0.143:7770/health` → `{"status":"ok"}`.
 - `/api/models` reports `n_ctx:262144, n_ctx_train:262144, n_params:27.3B`.
 - Model load is slow (a few minutes cold for the ~30 GB payload with a fresh
  KV cache). Health returns 503 while loading. Do not kill the process.
  `deploy.sh` waits up to 20 minutes (see `LLM_WAIT`).
 - Kernel spam about `amdgpu: SVM mapping failed` is the known documented
  symptom, not root cause (see the crash-loop procedure in the project
  AGENTS.md). Investigate other causes before assuming memory pressure.
 - A client abort logs `W srv stop: cancel task, id_task = N`; a natural stop
  logs only `slot release` with `truncated = 0`.
 ## Env tunables
 `scripts/frank-default-large.env`. Serving: `CTX_SIZE=262144`,
 `PARALLEL=1`, `BATCH_SIZE=512`, `UBATCH_SIZE=512`, `NUM_THREADS=16`,
 `THREAD_BATCH=32`, `CACHE_TYPE_K=f16`, `CACHE_TYPE_V=turbo4`,
 `CTX_CHECKPOINTS=32`, `CACHE_RAM=8192`. Reasoning: `REASONING=on`,
 `REASONING_FORMAT=none`. Sampling: `TEMPERATURE=0.7`, `TOP_K=20`,
 `TOP_P=0.8`. MTP: `MTP_N_MAX=4`, `MTP_N_MIN=0`, `MTP_P_MIN=0.0`,
 `MTP_P_SPLIT=0.10`. Strix Halo: `HSA_OVERRIDE_GFX_VERSION=11.5.1`,
 `GGML_HIP_ENABLE_UNIFIED_MEMORY=1`.
 **Never name a tunable `TEMP`, `HOST`, or `GPU_MAX_HEAP_SIZE`** — the ROCm HIP
 runtime reads those bare vars (see the 2026-08-13 TEMP incident). llama.cpp
 reads `--host/--port/--temp` as CLI flags, never as env vars.
 ## History
 - **2026-08-25 (today):** switched the large service to the kingjones
  Q8_0-AGENT model with 262144 context and reasoning on. Downloaded the three
  payload files from the HF repo, raised the TTM/GTT to 114 GiB for the full
  context, and persisted the env/serve script in this repo.
 - Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
  retired the same day for the AGENT model). See [[kairic-edge]].
@@ -15,10 +15,13 @@ timestamp: 2026-08-24
 # kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
-> **Active 2026-08-24.** `frank-default-large` serves this model. It
+> **Retired 2026-08-25.** The kingjones ROCmFP4 STRIX MTP payload has been
-> replaced the jcbtc v3 CIRU ActiveFPX PromptForge payload on the same
+> replaced by the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on the same
-> ROCmFPX runtime and the same CIRU llama.cpp binary. See
+> port and iGPU, under a new runtime (charlie12345/ROCmFPX ROCmI4 W4A4). See
-> [[ciru-activefpx-promptforge]] for the retired v3 page.
+> [[eemin-rocmi4-mtp]]. The old kingjones GGUF + standalone MTP Q4_0 draft are
 > retired on this slot. The CIRU ROCmFPX runtime/binary is also retired here.
 > _This page documents the retired model and its runbook for reference._
 `frank-default-large` (port 7770) serves
 `Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
@@ -19,7 +19,7 @@ Related pages:
 | 2 | MTP `n_max` explicit 4–5 + Q4_0 draft head | +5–15% on the spec share | restart, 1.56 GB file | test 08-16 |
 | 3 | Plain `Q8_0` A/B vs UD-Q8_K_XL (same 8 bits) | dequant ALUs may differ | restart, 29 GB file | test 08-16 |
 | 4 | Prefill path: ROCm backend + tuned ubatch/threads-batch | 2–3× prefill for agentic turns | per-workload backend choice | partial data |
-| 5 | TTM `pages_limit` raise (15.2 GiB → 90+) | removes a hard VRAM ceiling, stable 96 GiB budget | reboot | pending |
+| 5 | TTM `pages_limit` raised to 114 GiB | removes the hard VRAM ceiling; the full 262k f16 KV plus the Q8 AGENT payload, MTP draft, and mmproj fit in one slot | modprobe, set live 2026-08-25 | applied (29,883,536 pages) |
 | 6 | Vulkan ICD pinning + loader-layer disable | robustness; small | service env vars | test 08-16 |
 Everything else (quantization downgrade, NVFP4, larger drafts) is documented
@@ -166,11 +166,13 @@ hybrid state, upstream). Levers:
 ### 5. OS level (from the Improvements doc, re-verified 2026-08-15)
- **TTM `pages_limit` is still 3,995,281 pages ≈ 15.24 GiB** — verified
+- **TTM `pages_limit` is now 114 GiB (29,883,536 pages)** — applied 2026-08-25.
-  today, unchanged. The 96 GiB GTT budget only materializes if the TTM cap
+  Written live to `/sys/module/ttm/parameters/pages_limit` (no reboot required)
-  is raised (kernel parameter, needs reboot). Until then the effective
+  and persisted in `/etc/modprobe.d/ttm.conf`. The full 262k f16 KV cache plus
-  ceiling is lower and the 28 GiB load-stall risk remains. **Highest-value
+  the ~28 GB Q8 AGENT payload, MTP draft, and mmproj now fit in one slot. The
-  single OS change; schedule with the next reboot.**
+  load-stall risk from the old 15 GiB cap is resolved. Note: 96 GiB is the
  driver-reported VRAM carve-out (VGM/SVM pool); the TTM pages_limit is the
  kernel page map that was the real ceiling.
 - `iommu=pt` — verified present on the kernel cmdline. Done.
 - **Vulkan ICD pinning** — `radeon_icd.json` exists; no `VK_ICD_FILENAMES`
  or loader-layer env is set. Pinning the radeon ICD and disabling loader
@@ -51,6 +51,7 @@ covers both workloads).
 ## Dankclaw Services
 - [[SearXNG]] — privacy metasearch (VPN-routed)
 - [[SearXNG Providers]] — unblocking Google & other upstream engines behind the VPN
 - [[Gluetun]] — the VPN tunnel container
 - [[Valkey]] — SearXNG cache and limiter store
 - [[ComfyUI]] — node-based AI image generation (CPU)
@@ -96,6 +97,9 @@ The LLM inference fleet on the two hosts. Full index:
 - [[deployment]] — unified deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
 - [[eemin-rocmi4-mtp]] — the caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP model on Dankclaw (frank-default-large, active)
 - [[kingjones-strix-mtp]] — the RETIRED kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (superseded 2026-08-25)
 - [[kingjones-q8agent-rocmfpx]] — the Q8_0-AGENT runbook for frank-default-large (active): serving, download, 262144 context, reasoning + preserve_thinking, 114 GiB TTM/GTT
 - [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
 - [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
 - [[dealign-ornith-crack]] — the dealign Ornith-1.5-35B-A3B CRACK model on Dankserver (active)
@@ -0,0 +1,218 @@
 #!/usr/bin/env bash
 # bench-ubatch-rocmi4.sh — sweep ubatch size at 8k context on the
 # frank-default-large ROCmI4 embedded-MTP stack (dankclaw, gfx1151).
 #
 # Boots one llama-server instance per ubatch on a scratch port with a fixed
 # -b and -c 8192, runs a standardized ~75%-of-ctx prose workload, and reads
 # authoritative prefill/generation tok/s from the server's own print_timing
 # lines. Steps from UBATCH_MIN to UBATCH_MAX inclusive by UBATCH_STEP.
 #
 # Usage:
 #   ./scripts/bench-ubatch-rocmi4.sh
 #   UBATCH_MIN=512 UBATCH_MAX=4096 UBATCH_STEP=512 CTX=8192 RUNS=3 \
 #     ./scripts/bench-ubatch-rocmi4.sh
 #
 # Benchmarking policy: stop the production service first so the GPU is not
 # starved. With STOPSVC=1 the script restarts frank-default-large on exit.
 set -euo pipefail
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}"
 if [[ -f "$ENV_FILE" ]]; then
    set -a
    # shellcheck disable=SC1090
    source "$ENV_FILE"
    set +a
 fi
 # --- sweep parameters -------------------------------------------------------
 : "${UBATCH_MIN:=512}"
 : "${UBATCH_MAX:=4096}"
 : "${UBATCH_STEP:=512}"
 : "${CTX:=8192}"                 # 8k contexts
 : "${BENCH_BATCH:=$CTX}"         # fixed batch = ctx, so the full 8k prompt fits
 : "${OUT_TOKENS:=256}"
 : "${RUNS:=3}"                   # decode runs per ubatch
 : "${PORT_BASE:=7894}"           # scratch ports 7894..
 : "${STOPSVC:=0}"                # stop/restart the live unit for a clean GPU
 # --- resolved paths (fall back to serve-script defaults) --------------------
 : "${LLAMA_BIN:=/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server}"
 : "${MODEL_DIR:=/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP}"
 : "${MODEL_FILE:=Qwen3.8-27B-Q4_0_ROCMI4.gguf}"
 : "${MMPROJ_FILE:=mmproj-BF16.gguf}"
 : "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
 : "${ALIAS:=frank-default-large}"
 : "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
 : "${GGML_HIP_ENABLE_UNIFIED_MEMORY:=1}"
 : "${MTP_N_MAX:=16}"
 : "${MTP_P_MIN:=0.60}"
 MODEL="$MODEL_DIR/$MODEL_FILE"
 MMPROJ="$MODEL_DIR/$MMPROJ_FILE"
 for f in "$MODEL" "$CHAT_TEMPLATE_FILE"; do
    if [[ ! -f "$f" ]]; then
        echo "missing required file: $f" >&2
        exit 1
    fi
 done
 if [[ ! -x "$LLAMA_BIN" ]]; then
    echo "missing llama-server binary: $LLAMA_BIN" >&2
    exit 1
 fi
 MMPROJ_ARGS=()
 if [[ -f "$MMPROJ" ]]; then
    MMPROJ_ARGS=(--mmproj "$MMPROJ")
 fi
 STopped=0
 cleanup() {
    if [ "$STopped" = 1 ]; then
        echo "=== restarting frank-default-large ==="
        sudo systemctl restart frank-default-large || true
        STopped=0
    fi
 }
 trap cleanup EXIT
 stop_live_service() {
    if [ "$STOPSVC" = 1 ]; then
        echo "=== stopping frank-default-large for a clean GPU ==="
        sudo systemctl stop frank-default-large || true
        STopped=1
    fi
 }
 # wait_health <pid> <port> — poll /health until ready or the server dies.
 wait_health() {
    local pid="$1" port="$2" i ready=0 crashed=0
    for i in $(seq 1 900); do
        if ! kill -0 "$pid" 2>/dev/null; then crashed=1; break; fi
        if curl -sf "http://127.0.0.1:${port}/health" >/dev/null 2>&1; then
            ready=1; break
        fi
        [ $((i % 10)) = 0 ] && echo -n .
        sleep 2
    done
    [ "$crashed" = 1 ] && { echo " BACKEND_CRASHED (before ready)"; return 2; }
    [ "$ready" = 1 ] || { echo " FAIL (timeout)"; return 1; }
    echo " [ready]"
    return 0
 }
 # run_one <ubatch> <port> <out_tokens> <runs> — boot one server instance, run
 # the workload, parse print_timing, print a RESULT line.
 run_one() {
    local ubatch="$1" port="$2" out_tokens="$3" runs="$4"
    local log="/tmp/ubatch-${ubatch}-p${port}.log"
    : > "$log"
    echo "=== ubatch=${ubatch} ctx=${CTX} port=${port} runs=${runs} ==="
    "$LLAMA_BIN" \
        --model "$MODEL" \
        "${MMPROJ_ARGS[@]}" \
        --chat-template-file "$CHAT_TEMPLATE_FILE" \
        --chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
        --alias bench-ubatch \
        --host 127.0.0.1 --port "$port" \
        -dev ROCm0 -ngl 999 -np 1 \
        -t 16 -tb 32 \
        -c "$CTX" -b "$BENCH_BATCH" -ub "$ubatch" \
        -fa on -ctk f16 -ctv f16 -ctxcp 16 \
        --cache-ram 8192 --cache-prompt --metrics --no-mmap \
        --spec-type draft-mtp --spec-mtp-strict-qwen \
        --spec-draft-device ROCm0 --spec-draft-ngl all \
        --spec-draft-type-k f16 --spec-draft-type-v f16 \
        --spec-draft-n-max "$MTP_N_MAX" --spec-draft-n-min 0 \
        --spec-draft-p-min "$MTP_P_MIN" \
        --spec-draft-backend-sampling \
        --temp 0.7 --top-p 0.95 \
        --reasoning on --reasoning-format deepseek --reasoning-budget -1 \
        -lv 4 >>"$log" 2>&1 &
    local pid=$!
    wait_health "$pid" "$port" || { tail -30 "$log"; return 2; }
    # one python pass per run; reuse the same boot across runs
    local r
    for r in $(seq 1 "$runs"); do
        echo "--- run ${r}/${runs} ---"
        python3 - "$ubatch" "$port" "$out_tokens" "$CTX" "$log" <<'PY'
 import sys, time, json, urllib.request
 ubatch, port, out_tokens, ctx, log = sys.argv[1], sys.argv[2], int(sys.argv[3]), int(sys.argv[4]), sys.argv[5]
 # Flowing prose at ~75% of ctx so MTP acceptance is realistic.
 VAR = (
    "narrative nonfiction blends observation with reflection, ",
    "and a careful writer lets each clause earn its place ",
    "in the sentence's rising line, so meaning accumulates slowly. ",
    "The reader is not hurried; the reader is led. Loose and periodic, ",
    "the style turns on concrete nouns and verbs that carry weight, ",
    "and it resists the abstract noun that hides a vague claim. ",
    "A paragraph is not a list. It is a single turn of thought that ",
    "opens, develops, and lands. This is the craft of prose. Choose, ",
    "arrange, and above all omit. Omission is the discipline that ",
    "keeps the whole clear, and clarity is the courtesy we owe the ",
    "willing reader."
 )
 n = len(VAR)
 parts = []
 t = 0
 while t * 76 < (ctx * 3) // 4:
    for k in range(5):
        parts.append(VAR[(t + k) % n])
    parts.append(f" This is cued by note {t}. ")
    t += 1
 prompt = "".join(parts)
 prompt += ("\n\nNow, in a detailed and flowing paragraph, continue this essay about "
           "the philosophy of language and its influence on modern computation. ")
 body = json.dumps({
    "prompt": prompt, "max_tokens": out_tokens,
    "temperature": 0.0, "stream": False,
 }).encode()
 req = urllib.request.Request(f"http://127.0.0.1:{port}/v1/completions",
                             data=body, headers={"Content-Type": "application/json"})
 t0 = time.time()
 with urllib.request.urlopen(req, timeout=7200) as resp:
    out = json.load(resp)
 dt = time.time() - t0
 pt = out.get("usage", {}).get("prompt_tokens", 0)
 ct = out.get("usage", {}).get("completion_tokens", 0)
 lines = open(log, encoding="utf-8", errors="ignore").read().splitlines()
 prefill = [l for l in lines if "prompt eval time" in l]
 evals = [l for l in lines if "eval time" in l and "prompt eval" not in l]
 def grab(arr):
    return arr[-1].strip() if arr else "n/a"
 g_tps = ""
 if evals and ct:
    # eval time line: "eval time = .. ms for N tokens"
    ev = evals[-1].split("= ")[-1]
    ms = float(ev.split(" for")[0].strip())
    g_tps = f"{ct / (ms / 1000):.2f}"
 print(f"== RESULT ubatch={ubatch} run={r} "
      f"PREFILL={grab(prefill)} GEN_TPS={g_tps} "
      f"pt={pt} ct={ct} wall={dt:.1f}s")
 PY
    done
    kill "$pid" 2>/dev/null || true
    wait "$pid" 2>/dev/null || true
 }
 main() {
    stop_live_service
    local idx=0 ubatch port
    printf "%-10s %-14s %-10s\n" "UBATCH" "PREFILL" "GEN_TPS"
    for ubatch in $(seq "$UBATCH_MIN" "$UBATCH_STEP" "$UBATCH_MAX"); do
        port=$((PORT_BASE + idx))
        run_one "$ubatch" "$port" "$OUT_TOKENS" "$RUNS"
        idx=$((idx + 1))
    done
    echo "=== sweep complete ==="
 }
 main "$@"
@@ -0,0 +1,63 @@
 #!/usr/bin/env bash
 # Download the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT payload for
 # frank-default-large from HuggingFace into /home/dank/models/.../Q8_0-AGENT.
 #
 # Uses the token in ~/.cache/huggingface/token so the ~28 GB main file does not
 # hit the anonymous 10 GB/30 min rate limit. Run on Dankclaw.
 #
 #   bash scripts/download-q8agent.sh
 #   bash scripts/download-q8agent.sh --verify-only
 set -euo pipefail
 REPO="kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF"
 DIR="/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT"
 EXPECTED=(
  "Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf:28214425952"
  "mmproj-Qwen3.8-27B-BF16.gguf:931145888"
  "mtp-Qwen3.8-27B-Q4_0.gguf:1680271648"
 )
 cd -- "$(dirname -- "$0")/.."
 main() {
  local mode="download"
  [[ "${1:-}" == "--verify-only" ]] && mode="verify"
  mkdir -p "$DIR"
  cd "$DIR"
  local token
  token="$(cat "$HOME/.cache/huggingface/token")"
  [[ -n "$token" ]] || { echo "no HF token in $HOME/.cache/huggingface/token" >&2; exit 1; }
  dl() {
    local f="$1"
    curl -fsSL -C - -o "$f" --retry 5 --retry-delay 3 --retry-max-time 3600 \
      -H "Authorization: Bearer $token" \
      "https://huggingface.co/$REPO/resolve/main/$f"
    echo "  done: $f ($(du -h "$f" | cut -f1))"
  }
  if [[ "$mode" == "download" ]]; then
    echo "downloading $REPO into $DIR"
    dl "Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf"
    dl "mmproj-Qwen3.8-27B-BF16.gguf"
    dl "mtp-Qwen3.8-27B-Q4_0.gguf"
  fi
  echo "verifying sizes"
  local ok=1 f want act
  for entry in "${EXPECTED[@]}"; do
    f="${entry%%:*}"; want="${entry##*:}"
    act="$(stat -c%s "$f")"
    if [[ "$act" -eq "$want" ]]; then
      echo "  OK   $f ($act bytes)"
    else
      echo "  BAD  $f ($act bytes, want $want)" >&2
      ok=0
    fi
  done
  [[ "$ok" -eq 1 ]] || exit 1
  echo "all files present and sized correctly"
 }
 main "$@"
@@ -1,75 +1,72 @@
-# Tunables for frank-default-large (CIRU llama.cpp, dankclaw, Strix Halo
+# Tunables for frank-default-large (kingjones Qwen3.8-27B ROCmFPX Q8 AGENT,
-# gfx1151).
+# dankclaw, Strix Halo gfx1151).
 #
-# Runtime: the PromptForge llama-server built from ciru-ai/ROCmFPX tag
+# Runtime: ROCmFPX llama.cpp with TurboQuant support. Serves the kingjones
-# qwen3.8-activefpx-promptforge-v2.3 plus the v3 output-K8 runtime patch
+# AGENT GGUF plus its standalone MTP draft and vision projector. Alias
-# serving kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4 (dense, Q6 AGENT
+# frank-default-large, port 7770.
 # GGUF) with the standalone MTP Q4_0 draft head. Alias frank-default-large,
 # port 7770.
 #
 # This model replaced jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (v3) on
 # 2026-08-24. The new model is a plain ROCmFP4 GGUF (no PromptForge .pfs
 # sidecars, no Output-K8 proxy); the requested file is Q6_.
 # WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE. The ROCm
 # HIP runtime reads those bare env vars (see the 2026-08-13 TEMP incident).
 # llama.cpp reads --host/--port/--temp as CLI flags, never as env vars.
-# CIRU llama-server binary (ROCmFPX v2.3 + v3 output-K8 patch build, built
+# ROCmFPX build with TurboQuant KV-cache support.
-# against the private TheRock ROCm 7.15 SDK kept in the retired ling3-ciru
+LLAMA_BIN=/home/dank/ciru-qwen38/ROCmFPX/build-kairic/bin/llama-server
 # venv). ROCm 7.15 is required for the advertised Strix Halo performance.
 LLAMA_BIN=/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server
-# Model files. Plain dense Qwen3.8-27B ROCmFP4 Q6 AGENT GGUF plus the standalone
+# Model files from https://huggingface.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF.
-# MTP draft head and the vision projector. Kingjones 2026-08-24.
+MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT
-MODEL_DIR=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP
+MODEL_FILE=Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf
-MODEL_FILE=Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf
+MTP_DRAFT_FILE=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf
-MTP_DRAFT_FILE=mtp-Qwen3.8-27B-Q4_0.gguf
+MMPROJ_FILE=mmproj-Qwen3.8-27B-BF16.gguf
 MMPROJ_FILE=mmproj-Qwen3.8-27B-Q8_0.gguf
-# Qwen-Sharp chat template (tracked in the repo, staged on the host).
+# Qwen-Sharp chat template (tracked in the repo, staged on the host). The Sharp
 # template drives thinking on by default (medium effort) and produces the
 # peg-native + deepseek chat/reasoning shape the Pi agent expects.
 CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/chat-template.jinja
-# enable_thinking + preserve_thinking on by default in the Sharp template;
+CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}'
 # medium effort default. preserve_thinking keeps <think> blocks in recorded
 # assistant messages (set to false to strip them, true to always keep them).
 CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true}'
 # ROCm 7.15 SDK paths (inside the retired ling3-ciru venv).
 ROCM_DEVEL=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_devel
 ROCM_LIB=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_libraries
 ROCM_CORE=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_core
 # Alias + network.
 ALIAS=frank-default-large
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7770
-# Serving: 196608 (192K) context, 1 slot. f16 KV, context checkpoints,
+# Serving: 262144 context (the model's maximum), 1 slot. f16 KV, context
-# prompt cache, draft-mtp with standalone Q4_0 head, n_max 4 (the knee for
+# checkpoints, prompt cache, standalone MTP draft on ROCm0. The full 262144
-# the dense model). Unified memory; rocm-smi reports the 64 GiB budget.
+# context KV cache plus the ~28-30 GB Q8 model + MTP draft + mmproj fit in the
-CTX_SIZE=196608
+# lifted 114 GiB TTM/GTT budget on the Strix Halo (see the GTT 114 GiB change).
 # Concurrent requests queue; the server never preempts a running generation.
 CTX_SIZE=262144
 PARALLEL=1
-BATCH_SIZE=8192
+# Batch/ubatch follow the existing qualified Dankclaw profile.
-UBATCH_SIZE=8192
+BATCH_SIZE=512
 UBATCH_SIZE=512
 NUM_THREADS=16
 THREAD_BATCH=32
 CACHE_TYPE_K=f16
-CACHE_TYPE_V=f16
+CACHE_TYPE_V=turbo4
 CTX_CHECKPOINTS=32
 CACHE_RAM=8192
-# Dense-model MTP draft (standalone Q4_0 head, n_max 4). No PromptForge .pfs
+# Standalone MTP draft head, loaded from MTP_DRAFT_FILE by the serve script.
-# sidecars are loaded; the LLAMA_CPU_ARGMAX fast path stays off.
+MTP_N_MAX=4
-SPEC_DRAFT_N_MAX=4
+MTP_N_MIN=0
-SPEC_DRAFT_N_MIN=0
+MTP_P_MIN=0.0
-SPEC_DRAFT_P_MIN=0.0
+MTP_P_SPLIT=0.10
 SPEC_DRAFT_P_SPLIT=0.10
-# Sampling (kingjones model card / Sharp default).
+# Sampling (Sharp default).
-TEMPERATURE=1.0
+TEMPERATURE=0.7
 TOP_K=20
-TOP_P=0.95
+TOP_P=0.8
 MIN_P=0.0
 PRESENCE_PENALTY=1.5
-# Thinking on (Qwen-Sharp template drives thinking on; reasoning format
+# Reasoning on (Sharp template drives it; preserve_thinking + preserve_reasoning
-# projects the tar into reasoning_content).
+# keep the reasoning trail in the conversation). reasoning-format none lets the
 # template project the reasoning content into reasoning_content.
 REASONING=on
-REASONING_FORMAT=deepseek
+REASONING_FORMAT=none
 # Strix Halo unified memory + native gfx1151 detection for this ROCmFPX build.
 HSA_OVERRIDE_GFX_VERSION=11.5.1
 GGML_HIP_ENABLE_UNIFIED_MEMORY=1
 VERBOSITY=4
@@ -1,5 +1,5 @@
 [Unit]
-Description=CIRU llama.cpp Qwen3.8-27B ROCmFP4 STRIX MTP + Sharp template (dankclaw, frank-default-large)
+Description=Kairic Edge ROCmFPX llama.cpp Qwen3.8-27B IU4 + MTP (dankclaw, frank-default-large)
 After=network-online.target
 Wants=network-online.target
@@ -1,28 +1,9 @@
 #!/usr/bin/env bash
-# Serve kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP (dense) as
+# Serve the kingjones Qwen3.8-27B ROCmFPX Q8 AGENT model as frank-default-large.
-# frank-default-large via the CIRU ROCmFPX llama.cpp runtime on Dankclaw
+# The deployment uses a standalone MTP draft head and Turbo4 V-side KV cache.
 # (Strix Halo gfx1151).
 #
 # Runtime: the PromptForge llama-server built from ciru-ai/ROCmFPX tag
 # qwen3.8-activefpx-promptforge-v2.3 (commit 103af8b41) plus the v3 output-K8
 # runtime patch (commit af0eac6), against the private TheRock ROCm 7.15 SDK
 # (kept in the retired ling3-ciru venv). The model is the plain ROCmFP4
 # Q6 AGENT GGUF (no PromptForge .pfs sidecars) with the standalone MTP draft
 # head mtp-Qwen3.8-27B-Q4_0.gguf. The Qwen-Sharp chat template is applied at
 # runtime.
 #
 # Spec: dense model -> native MTP draft at n_max 4 (the knee, per the model
 # card). The draft head is a separate Q4_0 GGUF loaded via --model-draft.
 #
 # The model card requires ROCm 7.15 (not the system ROCm 7.2.4) for the
 # advertised Strix Halo performance. LD_LIBRARY_PATH therefore points at the
 # 7.15 SDK libraries first.
 #
 # Tunables come from frank-default-large.env in this directory unless
 # FRANK_LARGE_ENV_FILE overrides that path.
 set -euo pipefail
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
 ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-default-large.env}"
 if [[ -f "$ENV_FILE" ]]; then
    set -a
@@ -31,102 +12,73 @@ if [[ -f "$ENV_FILE" ]]; then
    set +a
 fi
-# --- defaults (used when the env file does not set them) -------------------
+: "${LLAMA_BIN:=/home/dank/ciru-qwen38/ROCmFPX/build-kairic/bin/llama-server}"
-: "${LLAMA_BIN:=/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server}"
+: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-27B-ROCMFPX-Q8_0-AGENT}"
-: "${MODEL_DIR:=/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP}"
+: "${MODEL_FILE:=Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf}"
-: "${MODEL_FILE:=Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf}"
+: "${MTP_DRAFT_FILE:=$MODEL_DIR/mtp-Qwen3.8-27B-Q4_0.gguf}"
-: "${MTP_DRAFT_FILE:=mtp-Qwen3.8-27B-Q4_0.gguf}"
+: "${MMPROJ_FILE:=mmproj-Qwen3.8-27B-BF16.gguf}"
 : "${MMPROJ_FILE:=mmproj-Qwen3.8-27B-Q8_0.gguf}"
 : "${CHAT_TEMPLATE_FILE:=$SCRIPT_DIR/chat-template.jinja}"
 : "${ROCM_DEVEL:=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_devel}"
 : "${ROCM_LIB:=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_libraries}"
 : "${ROCM_CORE:=/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/_rocm_sdk_core}"
 : "${ALIAS:=frank-default-large}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7770}"
 : "${CTX_SIZE:=262144}"
 : "${PARALLEL:=1}"
-: "${BATCH_SIZE:=4096}"
+: "${BATCH_SIZE:=2048}"
-: "${UBATCH_SIZE:=4096}"
+: "${UBATCH_SIZE:=512}"
 : "${NUM_THREADS:=16}"
 : "${THREAD_BATCH:=32}"
 : "${CACHE_TYPE_K:=f16}"
-: "${CACHE_TYPE_V:=f16}"
+: "${CACHE_TYPE_V:=turbo4}"
-: "${CTX_CHECKPOINTS:=16}"
+: "${CTX_CHECKPOINTS:=32}"
 : "${CACHE_RAM:=8192}"
-: "${SPEC_DRAFT_N_MAX:=4}"
+: "${MTP_N_MAX:=4}"
-: "${SPEC_DRAFT_N_MIN:=0}"
+: "${MTP_N_MIN:=0}"
-: "${SPEC_DRAFT_P_MIN:=0.0}"
+: "${MTP_P_MIN:=0.0}"
-: "${SPEC_DRAFT_P_SPLIT:=0.10}"
+: "${MTP_P_SPLIT:=0.10}"
-: "${TEMPERATURE:=1.0}"
+: "${TEMPERATURE:=0.7}"
 : "${TOP_K:=20}"
-: "${TOP_P:=0.95}"
+: "${TOP_P:=0.8}"
-: "${REASONING:=on}"
+: "${MIN_P:=0.0}"
-: "${REASONING_FORMAT:=deepseek}"
+: "${PRESENCE_PENALTY:=1.5}"
-: "${CHAT_TEMPLATE_KWARGS:={}}"
+: "${REASONING:=off}"
-: "${VERBOSITY:=4}"
+: "${REASONING_FORMAT:=none}"
 : "${HSA_OVERRIDE_GFX_VERSION:=11.5.1}"
 MODEL="$MODEL_DIR/$MODEL_FILE"
-MTP_DRAFT="$MODEL_DIR/$MTP_DRAFT_FILE"
+for f in "$LLAMA_BIN" "$MODEL" "$MTP_DRAFT_FILE" "$MODEL_DIR/$MMPROJ_FILE" "$CHAT_TEMPLATE_FILE"; do
-
+    if [[ ! -r "$f" ]]; then
-# Plain dense model: no PromptForge .pfs sidecars, no Output-K8 proxy, no IU4
+        echo "missing or unreadable required artifact: $f" >&2
 # lane. The v3 MTP is driven by the standalone Q4_0 draft head through the
 # standard draft-mtp path. Unset any leftover PromptForge env lanes.
 unset PROMPTFORGE_SIDECAR PROMPTFORGE_GDN_SIDECAR PROMPTFORGE_MTP_OUTPUT_K8_PROXY || true
 unset PROMPTFORGE_MODE PROMPTFORGE_ENABLE_SMALLM_W8 || true
 unset PROMPTFORGE_MTP_OUTPUT_K8 PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE || true
 unset PROMPTFORGE_IU4_SIDECAR PROMPTFORGE_GDN_IU4_SIDECAR || true
 unset PROMPTFORGE_ENABLE_NGRAM_MOD PROMPTFORGE_ENABLE_NGRAM_M65_IU4 || true
 for f in "$MODEL" "$MTP_DRAFT" "$MODEL_DIR/$MMPROJ_FILE" "$CHAT_TEMPLATE_FILE"; do
    if [[ ! -f "$f" ]]; then
        echo "missing required file: $f" >&2
        exit 1
    fi
 done
 if [[ ! -x "$LLAMA_BIN" ]]; then
-    echo "missing llama-server binary: $LLAMA_BIN" >&2
+    echo "missing executable runtime: $LLAMA_BIN" >&2
    exit 1
 fi
 SERVER_DIR="$(cd -- "$(dirname -- "$LLAMA_BIN")" && pwd)"
 ROCM_PATH="${ROCM_PATH:-/opt/rocm}"
 LIBRARY_PATH="$SERVER_DIR:$ROCM_PATH/lib:$ROCM_PATH/lib/rocm_sysdeps/lib:$ROCM_PATH/lib/llvm/lib:$ROCM_PATH/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
-# ROCm 7.15 SDK libraries must precede any system ROCm in LD_LIBRARY_PATH.
+exec /usr/bin/env \
-export ROCM_PATH="$ROCM_DEVEL"
+    ROCM_PATH="$ROCM_PATH" \
-export LD_LIBRARY_PATH="${ROCM_LIB}/lib:${ROCM_CORE}/lib:${ROCM_DEVEL}/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+    HIP_VISIBLE_DEVICES=0 \
-
+    HSA_OVERRIDE_GFX_VERSION="$HSA_OVERRIDE_GFX_VERSION" \
-exec "$LLAMA_BIN" \
+    LD_LIBRARY_PATH="$LIBRARY_PATH" \
-    --model "$MODEL" \
+    GGML_CUDA_GRAPH_OPT=0 \
    LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=0 \
    LLAMA_MTP_CPU_ARGMAX_FASTPATH=1 \
    "$LLAMA_BIN" -m "$MODEL" --alias "$ALIAS" \
    --host "$BIND_ADDR" --port "$LISTEN_PORT" --jinja \
    -dev ROCm0 -ngl 999 -np "$PARALLEL" -c "$CTX_SIZE" \
    -b "$BATCH_SIZE" -ub "$UBATCH_SIZE" -fa on \
    -ctk "$CACHE_TYPE_K" -ctv "$CACHE_TYPE_V" -t "$NUM_THREADS" -tb "$THREAD_BATCH" \
    -ctxcp "$CTX_CHECKPOINTS" --cache-ram "$CACHE_RAM" --cache-prompt --cache-idle-slots --metrics \
    --spec-type draft-mtp --model-draft "$MTP_DRAFT_FILE" \
    --mmproj "$MODEL_DIR/$MMPROJ_FILE" \
-    --chat-template-file "$CHAT_TEMPLATE_FILE" \
+    --spec-draft-device ROCm0 --spec-draft-ngl 999 \
-    --chat-template-kwargs "$CHAT_TEMPLATE_KWARGS" \
+    --spec-draft-type-k "$CACHE_TYPE_K" --spec-draft-type-v "$CACHE_TYPE_V" \
-    --alias "$ALIAS" \
+    --spec-draft-n-max "$MTP_N_MAX" --spec-draft-n-min "$MTP_N_MIN" \
-    --host "$BIND_ADDR" \
+    --spec-draft-p-min "$MTP_P_MIN" --spec-draft-p-split "$MTP_P_SPLIT" \
-    --port "$LISTEN_PORT" \
+    --spec-draft-backend-sampling \
-    -dev ROCm0 \
+    --temp "$TEMPERATURE" --top-p "$TOP_P" --top-k "$TOP_K" --min-p "$MIN_P" \
-    -ngl 999 \
+    --presence-penalty "$PRESENCE_PENALTY" \
-    -c "$CTX_SIZE" \
+    --reasoning "$REASONING" --reasoning-format "$REASONING_FORMAT" --reasoning-budget -1
    -b "$BATCH_SIZE" \
    -ub "$UBATCH_SIZE" \
    -fa on \
    -ctk "$CACHE_TYPE_K" \
    -ctv "$CACHE_TYPE_V" \
    -np "$PARALLEL" \
    -ctxcp "$CTX_CHECKPOINTS" \
    --cache-ram "$CACHE_RAM" \
    --cache-prompt \
    --metrics \
    --no-mmap \
    --spec-type draft-mtp \
    --model-draft "$MTP_DRAFT" \
    --spec-draft-device ROCm0 \
    --spec-draft-ngl 999 \
    --spec-draft-type-k f16 \
    --spec-draft-type-v f16 \
    --spec-draft-n-max "$SPEC_DRAFT_N_MAX" \
    --spec-draft-n-min "$SPEC_DRAFT_N_MIN" \
    --spec-draft-p-min "$SPEC_DRAFT_P_MIN" \
    --spec-draft-p-split "$SPEC_DRAFT_P_SPLIT" \
    --temp "$TEMPERATURE" \
    --top-k "$TOP_K" \
    --top-p "$TOP_P" \
    --reasoning "$REASONING" \
    --reasoning-format "$REASONING_FORMAT" \
    --reasoning-budget -1 \
    -lv "$VERBOSITY"
@@ -0,0 +1,73 @@
 #!/usr/bin/env bash
 # Seam: frank-default-large model, ROCmFPX runtime, MTP, KV cache, template.
 #
 # load helpers
 load helpers
 ENV_FILE="$REPO_ROOT/scripts/frank-default-large.env"
 SERVE_FILE="$REPO_ROOT/scripts/serve-frank-large.sh"
 UNIT_FILE="$REPO_ROOT/scripts/frank-default-large.service"
 TEMPLATE_FILE="$REPO_ROOT/scripts/chat-template.jinja"
@test "frank-default-large serves the kingjones AGENT model" {
    grep -q '^MODEL_FILE=Qwen3\.8-27B-Q8_0_ROCMFPX_AGENT\.gguf$' "$ENV_FILE"
    grep -q '^MTP_DRAFT_FILE=' "$ENV_FILE"
    ! grep -q -- '--kairic-edge' "$SERVE_FILE"
 }
@test "frank-default-large uses the ROCmFPX runtime" {
    grep -q 'ROCmFPX/.*/bin/llama-server' "$ENV_FILE"
    grep -q 'build-kairic/bin/llama-server' "$ENV_FILE"
 }
@test "frank-default-large runs standalone MTP" {
    grep -q -- '--spec-type draft-mtp' "$SERVE_FILE"
    grep -qF -- '--model-draft "$MTP_DRAFT_FILE"' "$SERVE_FILE"
    grep -qF -- '--spec-draft-n-max "$MTP_N_MAX"' "$SERVE_FILE"
    grep -q '^MTP_N_MAX=4$' "$ENV_FILE"
 }
@test "frank-default-large has no obsolete PromptForge sidecar wiring" {
    ! grep -qE 'KAIRIC_|PROMPTFORGE_' "$SERVE_FILE"
    ! grep -qE 'KAIRIC_|PROMPTFORGE_' "$ENV_FILE"
 }
@test "frank-default-large fits the model and KV cache in unified memory" {
    grep -q '^CTX_SIZE=262144$' "$ENV_FILE"
    grep -q ': "${CTX_SIZE:=262144}"' "$SERVE_FILE"
 }
@test "frank-default-large has reasoning and preserved reasoning/thinking enabled" {
    grep -q '^REASONING=on$' "$ENV_FILE"
    grep -q 'enable_thinking":true' "$ENV_FILE"
    grep -q 'preserve_thinking":true' "$ENV_FILE"
    grep -q 'preserve_reasoning":true' "$ENV_FILE"
 }
@test "frank-default-large uses Turbo4 only for the V cache" {
    grep -q '^CACHE_TYPE_K=f16$' "$ENV_FILE"
    grep -q '^CACHE_TYPE_V=turbo4$' "$ENV_FILE"
    grep -qF -- '-ctk "$CACHE_TYPE_K" -ctv "$CACHE_TYPE_V"' "$SERVE_FILE"
 }
@test "frank-default-large exposes a swappable ubatch" {
    grep -q '^UBATCH_SIZE=512$' "$ENV_FILE"
    grep -q ': "${UBATCH_SIZE:=512}"' "$SERVE_FILE"
    grep -q -- '-ub "$UBATCH_SIZE"' "$SERVE_FILE"
 }
@test "frank-default-large uses the tracked Qwen-Sharp template on :7770" {
    grep -q '^CHAT_TEMPLATE_FILE=' "$ENV_FILE"
    grep -q -- '--jinja' "$SERVE_FILE"
    [ -s "$TEMPLATE_FILE" ]
    grep -q '^ALIAS=frank-default-large$' "$ENV_FILE"
    grep -q '^LISTEN_PORT=7770$' "$ENV_FILE"
    grep -q 'frank-default-large' "$UNIT_FILE"
 }
@test "frank-default-large env avoids the HIP TEMP/HOST collision" {
    # The 2026-08-13 incident: TEMP/HOST/GPU_MAX_HEAP_SIZE are HIP runtime vars.
    ! grep -qE '^(TEMP|HOST|GPU_MAX_HEAP_SIZE)=' "$ENV_FILE"
    grep -q '^TEMPERATURE=' "$ENV_FILE"
 }
Author	SHA1	Message	Date
Frank	54b2d84613	Upgrade frank-default-large to kingjones Q8_0 AGENT at full 262144 context frank-default-large now serves the kingjones Qwen3.8-27B-ROCmFPX-Q8_0-AGENT model with a standalone MTP draft head (Q4_0), f16 K + TurboQuant V KV cache, and the Sharp chat template. Context is maxed to 262144 (one slot); reasoning is on with preserve_thinking + preserve_reasoning in the chat kwargs. - frank-default-large.env: Q8_0 AGENT model paths, CTX_SIZE=262144, REASONING=on, REASONING_FORMAT=none, CHAT_TEMPLATE_KWARGS {enable_thinking, preserve_thinking, preserve_reasoning} - serve-frank-large.sh: standalone draft-mtp profile (already switched off the Kairic/PromptForge path) - download-q8agent.sh: reproducible HF fetch of the three payload files - docs/LLM/kingjones-q8agent-rocmfpx.md: Q8_0 AGENT runbook - AGENTS.md: topology + ACTIVE runtime section + TTM/GTT load note - qwen3.8-strix-optimizations, Dankclaw Improvements: TTM pages_limit now 114 GiB (applied 2026-08-25), full 262k KV + Q8 payload fit in one slot - frank_large_config.bats: assert 262144 context, reasoning + preserve_* enabled The TTM/GTT was raised to 114 GiB (29,883,536 pages) to hold the full 262k KV.	2026-08-25 07:12:02 -07:00
Frank	24efa98d56	docs: add SearXNG providers guide for unblocking Google behind Gluetun VPN SearXNG queries all ~50 upstream engines from one Gluetun VPN exit IP, so rate-sensitive engines (Google, DuckDuckGo) answer with reCAPTCHA/429 and SearXNG suspends the offending engine for hours-to-days, showing up as "unavailable". Add docs/Dankclaw/SearXNG Providers.md covering: the suspension model, the two failure modes (CAPTCHA vs selector drift), engine config (settings.yml engines: list — duplicate google, timeout, retry_on_http_error, JSON feed), request tuning (outgoing block, UA rotation, suspended_times — correcting the old rate_limit/useragents blog myths), routing/proxies (per-engine proxies, dedicated 2nd Gluetun, dedicated IP), and verification (/json, /config, check.searx.space). Crosslink into SearXNG.md and index.md.	2026-08-25 06:22:57 -07:00
Frank	b457457bfe	Switch large service to Kairic Edge v1.2	2026-08-25 04:41:04 -07:00
Frank	f86ed70c71	Use conservative four-token MTP drafts	2026-08-25 04:20:15 -07:00
Frank	3cf6353050	Reduce large model context to 64K	2026-08-25 04:13:00 -07:00
Frank	b0eb571321	Reduce large model context to 128K	2026-08-25 04:09:12 -07:00
Frank	ee3c3de750	Reduce frank-default-large context for KV headroom	2026-08-25 04:05:33 -07:00
Frank	7ca9dc248e	frank-default-large: swap to caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP Replace the retired kingjones ROCmFP4 STRIX MTP payload on Dankclaw with cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF (eemin namespace, public), served by the charlie12345/ROCmFPX llama.cpp build (ROCmI4 W4A4, c49ebdbd) on the Strix Halo iGPU. Q4_0_ROCMI4 with an embedded NextN/MTP head (no separate draft file); strict-Qwen MTP n16/p0.60/backend sampling on ROCm0. Binary /home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server; model dir /home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP. Updated serve script + env, ubatch bench, large-config bats test, docs, AGENTS.md.	2026-08-25 03:33:29 -07:00