frank-large: replace stock llama.cpp with gufo (gufo-org/gufo) on Dankclaw

gufo is a vertical inference engine built for exactly this hardware
(Ryzen AI MAX+ 395 / Radeon 8060S, gfx1151), and Dankclaw's frank-large
has swapped runtimes seven times in search of a better one. This one
ships no prebuilt Linux binary, so the runtime is BUILT FROM SOURCE on
the host and the payload, port, alias and client contract stay exactly
where they are.

Runtime: gufo v0.1.1 (b0f8673) at /home/dank/GitPackages/gufo-v0.1.1,
built by the new scripts/build-gufo.sh. It links the host ROCm 7.2.4
through its RPATH — the opposite of the unsloth prebuilt it replaces,
which carried its own — so two host workarounds are encoded in that
script: Ubuntu's librocwmma-dev installs a truncated rocWMMA header set
that shadows AMD's and cannot be overridden with -I in HIP mode, and the
ROCm clang prefers Ubuntu 26.04's GCC 16 headers over the GCC 15 whose
libstdc++ is actually installed.

Shape is unchanged: one session, full 262144 ctx, the same UD-Q8_K_XL
target and the same DFlash2 drafter, greedy, text-only. Speculation
moves from llama.cpp's fixed depth 4 to gufo's adaptive block policy
capped at 7 proposals.

Client-contract deltas, all documented: the chat template is compiled
into the binary (no --chat-template-file, so the Qwen-Sharp 73k-token
failure mode is gone), reasoning_effort now takes the full OpenAI set
(so models.yml's reasoningEffortMap fold is no longer required — kept
anyway, so this commit does not also move every high role from xhigh to
high effort), and there is no KV-quant or prompt-cache-RAM flag.

The test seam now runs the serve script against a stub engine and
asserts on the argv it actually execs, instead of grepping the script's
text. The deploy handler gates on the runtime binary and returns before
it stops the serving unit.

Measured trade, from gufo's own Q8_K_XL tables against llama.cpp
b11069: prefill +37% to +80%, cold load 4.5x faster, decode +7% at zero
depth — but -14% at 65,536 tokens of depth and -24% at 131,072. On the
8-bit target the crossover is ~32k, and frank-large runs one 262,144
conversation. The Q4_K_XL target shows no such regression.

serverAppConfigs-docs/LLM/Dankclaw System Specs.md

@@ -101,8 +101,8 @@ cable between claw and server is required for 10G.
 
 - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
 - The GPU budget is the GTT aperture: ~114 GiB (`gtt_total` in `/sys/class/drm/card1/device/`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
-- The active service is `frank-large` on port 7770: stock llama.cpp (the unsloth b11030 prebuilt) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, q8_0 KV, DFlash2 drafter). See [[frank-large-unsloth-qwen38-27b-q8xl]].
-- The active runtime (the unsloth b11030 prebuilt) **bundles its own ROCm runtime** next to the binary (`RPATH=$ORIGIN/../lib`), so it has no host SDK gate — the system ROCm 7.2.4 is not on its path. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) is no longer in the serving path.
+- The active service is `frank-large` on port 7770: **gufo** (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source at `/home/dank/GitPackages/gufo-v0.1.1/`, serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, DFlash2 drafter). See [[frank-large-gufo-qwen38-27b]].
+- The active runtime **links against the host ROCm** (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH, so it needs the host HIP stack rather than a bundled one — the opposite of the retired llama.cpp prebuilt, which carried its own. Building it needs the ROCm dev packages that are installed, and two workarounds recorded in the runbook (drop `librocwmma-dev`; pin `--gcc-install-dir` to GCC 15). The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) is not in the serving path.
 - The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
 - 1.1 TB free disk space is ample for model storage.
 

serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md

@@ -0,0 +1,176 @@
+---
+type: reference
+title: "gufo Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)"
+description: "The active frank-large runbook — gufo (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source on Dankclaw and serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU: the from-source build and its two host workarounds, the one-session / full-window shape, what changed in the client contract versus llama.cpp, the measured prefill/decode trade, and the failure modes worth knowing."
+tags:
+  - llm
+  - gufo
+  - unsloth
+  - dankclaw
+  - frank-large
+  - strix-halo
+  - gfx1151
+  - q8
+  - dflash2
+  - rocm
+timestamp: 2026-09-28
+---
+
+# frank-large — gufo Qwen3.8-27B UD-Q8_K_XL (DFlash2)
+
+> **ACTIVE since 2026-09-28.** Replaces
+> [[frank-large-unsloth-qwen38-27b-q8xl]] (stock llama.cpp, the unsloth
+> b11030 prebuilt), which had replaced
+> [[frank-large-kairic-edge-qwen38-27b]]. Same host, same port (7770), same
+> alias (`frank-large`), **same weights**.
+
+## Identity
+
+- **Endpoint:** `frank-large` at `192.168.0.143:7770` (Dankclaw).
+- **Runtime:** [gufo](https://github.com/gufo-org/gufo) — "the Strix Halo
+  inference engine", a vertical engine built for exactly this hardware
+  (Ryzen AI MAX+ 395 / Radeon 8060S, `gfx1151`, unified memory). Pinned at
+  **v0.1.1** (commit `b0f8673`), built from source on the host to
+  `/home/dank/GitPackages/gufo-v0.1.1/`.
+- **Service shape:** root systemd unit `scripts/frank-large.service`
+  (`User=dank`, `LimitMEMLOCK=infinity`), launched by
+  `scripts/serve-frank-large.sh` with tunables from
+  `scripts/frank-large.env`.
+
+gufo ships **no prebuilt Linux binaries** (the v0.1.0/v0.1.1 release assets
+are empty) and no Nix on this host, so the runtime is a source build. It is
+also young: v0.1.0 landed 2026-09-28, the same day as this cutover.
+
+## Why this engine
+
+gufo's own measurement of the two engines on the same Qwen3.8-27B targets
+(Strix Halo, `628ed18e` vs llama.cpp `b11069`, greedy, DFlash2):
+
+| Q8_K_XL, DFlash2 | gufo | llama.cpp | delta |
+|---|---:|---:|---:|
+| prefill, depth 0 | 457.05 t/s | 333.85 t/s | **+36.9 %** |
+| prefill, depth 131,072 | 237.24 t/s | 131.80 t/s | **+80.0 %** |
+| decode, depth 0 (mixed) | 16.08 t/s | 15.00 t/s | +7.2 % |
+| decode, depth 32,768 (mixed) | 14.41 t/s | 14.44 t/s | −0.2 % |
+| decode, depth 65,536 (mixed) | 10.87 t/s | 12.62 t/s | **−13.9 %** |
+| decode, depth 131,072 (mixed) | 8.87 t/s | 11.62 t/s | **−23.7 %** |
+| cold load to HTTP-ready | 8.17 s | 36.47 s | **4.5× faster** |
+| peak memory @ 262,144 ctx | 50.58 GiB | 48.60 GiB | +4 % |
+
+Read that honestly: **prefill, load time and short-context decode improve;
+deep-context decode on the 8-bit target gets worse.** The crossover on Q8 is
+around 32k of cached depth. frank-large runs one 262,144-token conversation,
+so a long agent session spends its life on the losing side of that line
+while a short turn wins. The Q4_K_XL target does not show the regression
+(gufo is +11.7 % to +17.0 % at every depth) — that is the honest reason to
+revisit the quantization separately from the runtime.
+
+## Build
+
+`scripts/build-gufo.sh` reproduces the runtime end to end. It clones the
+pinned tag, configures the `release` preset (HIP, `--offload-arch=gfx1151`),
+builds, installs, and finishes with `gufo diagnose`. Two host facts it
+encodes, both found the hard way on 2026-09-28:
+
+- **Ubuntu's `librocwmma-dev` must not be installed.** It ships a truncated
+  rocWMMA header set (the public headers without `rocwmma/internal/`) into
+  `/usr/include`, where it shadows AMD's complete copy under
+  `/opt/rocm/include`. The first build died on `'internal/accessors.hpp'
+  file not found` from `/usr/include/rocwmma/rocwmma.hpp`, and **a `-I`
+  cannot fix it**: in HIP mode the host compilation pass searches the system
+  directories first. The package has no reverse dependencies; the script
+  removes it.
+- **`--gcc-install-dir` must point at GCC 15.** Ubuntu 26.04 has
+  `/usr/include/c++/16`, and the ROCm clang picks the newest tree, whose
+  `<atomic_wait.h>` references `std::__detail::__notify_impl` at a symbol
+  version the installed `libstdc++.so.6` (GLIBCXX_3.4.35) does not export.
+  The link then fails with `undefined symbol`. Pinning the headers to the
+  GCC the runtime library came from fixes it.
+
+Verify an installed runtime with `bash scripts/build-gufo.sh --verify`, or
+`gufo diagnose` directly — it reports `[PASS]` plus the detected
+architecture, driver and queue budget.
+
+## Shape
+
+- **Model:** `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL` — 27B dense,
+  29.30 GiB. **Unchanged by the cutover**, still fetched and sha256-verified
+  by `scripts/download-frank-large.sh`.
+- **Context:** one session, full 262,144 tokens (the model's native window),
+  `SESSIONS=1`. The fleet's single stream is unchanged.
+- **Spec method:** DFlash2, gufo's **adaptive** block policy
+  (`--draft-policy adaptive`, `--draft-tokens 7`). The drafter is the same
+  z-lab `Qwen3.8-27B-DFlash2-Q4_K_M` (1.07 GiB).
+- **Sampling:** greedy (`--temperature 0`), as every prior frank-large
+  served. gufo's per-model recommendation for Qwen3.8 thinking is *sampled*
+  (1.0 / 0.95 / top-k 20) — that is a separate decision, deliberately not
+  bundled into a runtime swap.
+- **Memory:** gufo's own table puts the 8-bit target at **50.58 GiB** peak
+  with context capacity 262,144, against the iGPU's ~114 GiB GTT aperture.
+
+## Client contract — what actually changed
+
+Everything the fleet depended on still holds: port 7770, alias `frank-large`,
+`reasoning_content`, `tool_calls`, streaming SSE, and text-only (gufo
+auto-discovers a `mmproj` beside the model; this payload has none).
+
+The differences worth knowing:
+
+- **No custom chat template.** gufo compiles the official Qwen Jinja into the
+  binary and takes no `--chat-template-file`. The Qwen-Sharp v22.4.0 file
+  this endpoint used with llama.cpp is **no longer in the command line** —
+  the 2026-09-21 "73,600-token prompt" incident cannot recur, because there
+  is no fallback template to land in. Thinking defaults to on at `xhigh`
+  with prior reasoning preserved (`--think auto --preserve-thinking on`,
+  spelled out in the serve script).
+- **`reasoning_effort` accepts the full OpenAI set** (`none`, `minimal`,
+  `low`, `medium`, `high`, `xhigh`, `max`). Qwen-Sharp rejected anything
+  outside `{xhigh, medium, low}` with an HTTP 500, which is why `models.yml`
+  still carries a `reasoningEffortMap` fold for this model. **The fold is
+  kept on purpose**: removing it would silently move every `high` role from
+  `xhigh` to `high` effort in the same commit as the runtime swap. Drop it as
+  its own change once the effort curve is a decision rather than a side
+  effect.
+- **No KV-cache quantization flag.** gufo picks its own KV dtype; there is
+  no `--cache-type-k`. The old `CACHE_RAM=8192` tunable is gone with it.
+  Prompt reuse is on by default, and `--cache-disk` (restart-safe) is opt-in
+  and currently off.
+- **Health/readiness split:** `GET /health` is process liveness and is what
+  the deployer gates on; `GET /ready` is model/backend readiness.
+  `GET /metrics` is Prometheus-format.
+- **Long bodies:** `--max-request-bytes` defaults to 8 MiB and unknown
+  request fields are rejected rather than ignored.
+
+## Failure modes worth knowing
+
+- **A missing runtime is a deploy error, not a restart loop.** The handler
+  gates on `LLM_LARGE_RUNTIME` *before* it stops anything, and the serve
+  script exits with a `build-gufo.sh` pointer rather than exec'ing.
+- **Host memory is the ceiling, and it is shared.** frank-large and ComfyUI
+  both hold gfx1151 compute queues. gufo reads
+  `/sys/class/kfd/kfd/proc/*/queues/*/type` and caps itself
+  (`GPU_MAX_HW_QUEUES`, 2 resident for `serve llm`) so a full device
+  time-slices instead of failing; four HIP processes is the device ceiling.
+- **Load is seconds, not minutes** (8 s measured cold), so a redeploy no
+  longer spends 20 minutes in `LLM_WAIT`.
+- **A v0.1.x runtime is a moving target.** Rebuilds are not automatic —
+  `RUNTIME_DIR` names a version, and the deployer will not silently
+  substitute another.
+
+## Rollback
+
+[[frank-large-unsloth-qwen38-27b-q8xl]] — the unsloth b11030 prebuilt is
+still at `/home/dank/GitPackages/unsloth-llama-b11030/` and the payload is
+the same directory the current runtime reads. Roll back by reverting
+`RUNTIME_DIR` (and the gufo-only tunables) in `scripts/frank-large.env`,
+restoring the serve script's llama.cpp command, and redeploying.
+
+# Citations
+
+- Runtime: https://github.com/gufo-org/gufo (pinned `v0.1.1`, MIT)
+- Measured tables: `docs/models/qwen3.8-27b/BENCHMARKS.md` in that repo
+- API contract: `docs/SERVER.md`; flags: `docs/CLI.md`
+- Build: `scripts/build-gufo.sh`; serve: `scripts/serve-frank-large.sh`;
+  tunables: `scripts/frank-large.env`
+- Payload: `scripts/download-frank-large.sh`
+- Predecessor: [[frank-large-unsloth-qwen38-27b-q8xl]]

serverAppConfigs-docs/LLM/frank-large-unsloth-qwen38-27b-q8xl.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: "stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)"
-description: "The active frank-large runbook — the stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, 31.46 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) and q8_0 KV on the Dankclaw Strix Halo iGPU: the prebuilt runtime, the full-window/one-stream shape, the DFlash2 spec method, the Qwen-Sharp template, the payload, the client contract, and the failure modes worth knowing."
+description: "The RETIRED frank-large runbook (2026-09-21 → 2026-09-28; replaced by [[frank-large-gufo-qwen38-27b]], and the rollback target) — the stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, 31.46 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) and q8_0 KV on the Dankclaw Strix Halo iGPU: the prebuilt runtime, the full-window/one-stream shape, the DFlash2 spec method, the Qwen-Sharp template, the payload, the client contract, and the failure modes worth knowing."
 tags:
   - llm
   - llama.cpp
@@ -19,10 +19,12 @@ timestamp: 2026-09-21
 
 # frank-large — stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2)
 
-> **ACTIVE since 2026-09-21.** Replaces [[frank-large-kairic-edge-qwen38-27b]]
-> (Kairic Edge IU4 W4A4, now the rollback target) and
-> [[frank-large-unsloth-qwen38-flash-next]] (the Flash-Next MoE it preceded).
-> Same port (7770), same alias (`frank-large`), same client contract surface.
+> **RETIRED 2026-09-28**, replaced by [[frank-large-gufo-qwen38-27b]] (the
+> gufo engine, same weights, same port and alias); the prebuilt is intact on
+> disk as the rollback target. ACTIVE from 2026-09-21, when it replaced
+> [[frank-large-kairic-edge-qwen38-27b]] (Kairic Edge IU4 W4A4) and
+> [[frank-large-unsloth-qwen38-flash-next]] (the Flash-Next MoE it
+> preceded).
 
 ## Identity
 

serverAppConfigs-docs/LLM/index.md

@@ -139,10 +139,18 @@ frank-small, port 7771]
   (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
 - [[deployment]] — deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
-- [[frank-large-unsloth-qwen38-27b-q8xl]] — ACTIVE frank-large runbook
-  since 2026-09-21 (root systemd unit, port 7770): stock llama.cpp
-  (unsloth b11030, ROCm gfx1151) serving
+- [[frank-large-gufo-qwen38-27b]] — ACTIVE frank-large runbook since
+  2026-09-28 (root systemd unit, port 7770): gufo (gufo-org/gufo) v0.1.1, the
+  Strix Halo inference engine, built from source on Dankclaw, serving
   unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense) with the z-lab DFlash2
+  Q4_K_M drafter (adaptive block policy) on the gfx1151 iGPU — one full
+  262,144-token conversation, greedy, text-only, the two from-source build
+  workarounds, the client-contract delta, the measured prefill/decode trade
+  against llama.cpp, troubleshooting
+- [[frank-large-unsloth-qwen38-27b-q8xl]] — RETIRED 2026-09-28 (replaced by
+  [[frank-large-gufo-qwen38-27b]]; rollback target): stock llama.cpp
+  (unsloth b11030, ROCm gfx1151) serving
+  unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the z-lab DFlash2
   Q4_K_M drafter (depth 4) on the gfx1151 iGPU — one full 262,144-token
   conversation, q8_0 KV (the build lacks fp8), text-only, Qwen-Sharp
   v22.4.0 template, the 114 GiB GTT memory model, client contract,

serverAppConfigs-docs/index.md

@@ -190,13 +190,21 @@ The LLM inference fleet on the two hosts. Full index:
   agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
   Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
   rollback record)
-- [[frank-large-unsloth-qwen38-27b-q8xl]] — the ACTIVE frank-large runbook
-  since 2026-09-21 (root systemd unit, port 7770): stock llama.cpp
+- [[frank-large-gufo-qwen38-27b]] — the ACTIVE frank-large runbook since
+  2026-09-28 (root systemd unit, port 7770): gufo (gufo-org/gufo) v0.1.1,
+  the Strix Halo inference engine, built from source on Dankclaw and
+  serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (a 27B dense model) with the
+  z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU — one 262,144-token
+  conversation, adaptive block policy, greedy, text-only, the two
+  from-source build workarounds, what changed in the client contract, the
+  measured prefill/decode trade against llama.cpp
+- [[frank-large-unsloth-qwen38-27b-q8xl]] — RETIRED 2026-09-28 (replaced by
+  [[frank-large-gufo-qwen38-27b]]; rollback target): stock llama.cpp
   (unsloth b11030, ROCm gfx1151) serving
-  unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (a 27B dense model) with the z-lab
-  DFlash2 Q4_K_M drafter (depth 4) on the gfx1151 iGPU — one 262,144-token
-  conversation, q8_0 KV (the build lacks fp8), text-only, Qwen-Sharp v22.4.0
-  template, the 114 GiB GTT memory model, client contract
+  unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the z-lab DFlash2 Q4_K_M drafter
+  (depth 4) on the gfx1151 iGPU — one 262,144-token conversation, q8_0 KV
+  (the build lacks fp8), text-only, Qwen-Sharp v22.4.0 template, the
+  114 GiB GTT memory model, client contract
 - [[frank-large-kairic-edge-qwen38-27b]] — RETIRED 2026-09-21 (replaced by
   [[frank-large-unsloth-qwen38-27b-q8xl]]; same-day swap): the Kairic Edge
   runtime (ciru-ai/ROCmFPX kairic-edge-qwen38-27b-v1.2, TheRock 7.15.0,