frank-large: replace stock llama.cpp with gufo (gufo-org/gufo) on Dankclaw

gufo is a vertical inference engine built for exactly this hardware
(Ryzen AI MAX+ 395 / Radeon 8060S, gfx1151), and Dankclaw's frank-large
has swapped runtimes seven times in search of a better one. This one
ships no prebuilt Linux binary, so the runtime is BUILT FROM SOURCE on
the host and the payload, port, alias and client contract stay exactly
where they are.

Runtime: gufo v0.1.1 (b0f8673) at /home/dank/GitPackages/gufo-v0.1.1,
built by the new scripts/build-gufo.sh. It links the host ROCm 7.2.4
through its RPATH — the opposite of the unsloth prebuilt it replaces,
which carried its own — so two host workarounds are encoded in that
script: Ubuntu's librocwmma-dev installs a truncated rocWMMA header set
that shadows AMD's and cannot be overridden with -I in HIP mode, and the
ROCm clang prefers Ubuntu 26.04's GCC 16 headers over the GCC 15 whose
libstdc++ is actually installed.

Shape is unchanged: one session, full 262144 ctx, the same UD-Q8_K_XL
target and the same DFlash2 drafter, greedy, text-only. Speculation
moves from llama.cpp's fixed depth 4 to gufo's adaptive block policy
capped at 7 proposals.

Client-contract deltas, all documented: the chat template is compiled
into the binary (no --chat-template-file, so the Qwen-Sharp 73k-token
failure mode is gone), reasoning_effort now takes the full OpenAI set
(so models.yml's reasoningEffortMap fold is no longer required — kept
anyway, so this commit does not also move every high role from xhigh to
high effort), and there is no KV-quant or prompt-cache-RAM flag.

The test seam now runs the serve script against a stub engine and
asserts on the argv it actually execs, instead of grepping the script's
text. The deploy handler gates on the runtime binary and returns before
it stops the serving unit.

Measured trade, from gufo's own Q8_K_XL tables against llama.cpp
b11069: prefill +37% to +80%, cold load 4.5x faster, decode +7% at zero
depth — but -14% at 65,536 tokens of depth and -24% at 131,072. On the
8-bit target the crossover is ~32k, and frank-large runs one 262,144
conversation. The Q4_K_XL target shows no such regression.

serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md

@@ -0,0 +1,176 @@
+---
+type: reference
+title: "gufo Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)"
+description: "The active frank-large runbook — gufo (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source on Dankclaw and serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU: the from-source build and its two host workarounds, the one-session / full-window shape, what changed in the client contract versus llama.cpp, the measured prefill/decode trade, and the failure modes worth knowing."
+tags:
+  - llm
+  - gufo
+  - unsloth
+  - dankclaw
+  - frank-large
+  - strix-halo
+  - gfx1151
+  - q8
+  - dflash2
+  - rocm
+timestamp: 2026-09-28
+---
+
+# frank-large — gufo Qwen3.8-27B UD-Q8_K_XL (DFlash2)
+
+> **ACTIVE since 2026-09-28.** Replaces
+> [[frank-large-unsloth-qwen38-27b-q8xl]] (stock llama.cpp, the unsloth
+> b11030 prebuilt), which had replaced
+> [[frank-large-kairic-edge-qwen38-27b]]. Same host, same port (7770), same
+> alias (`frank-large`), **same weights**.
+
+## Identity
+
+- **Endpoint:** `frank-large` at `192.168.0.143:7770` (Dankclaw).
+- **Runtime:** [gufo](https://github.com/gufo-org/gufo) — "the Strix Halo
+  inference engine", a vertical engine built for exactly this hardware
+  (Ryzen AI MAX+ 395 / Radeon 8060S, `gfx1151`, unified memory). Pinned at
+  **v0.1.1** (commit `b0f8673`), built from source on the host to
+  `/home/dank/GitPackages/gufo-v0.1.1/`.
+- **Service shape:** root systemd unit `scripts/frank-large.service`
+  (`User=dank`, `LimitMEMLOCK=infinity`), launched by
+  `scripts/serve-frank-large.sh` with tunables from
+  `scripts/frank-large.env`.
+
+gufo ships **no prebuilt Linux binaries** (the v0.1.0/v0.1.1 release assets
+are empty) and no Nix on this host, so the runtime is a source build. It is
+also young: v0.1.0 landed 2026-09-28, the same day as this cutover.
+
+## Why this engine
+
+gufo's own measurement of the two engines on the same Qwen3.8-27B targets
+(Strix Halo, `628ed18e` vs llama.cpp `b11069`, greedy, DFlash2):
+
+| Q8_K_XL, DFlash2 | gufo | llama.cpp | delta |
+|---|---:|---:|---:|
+| prefill, depth 0 | 457.05 t/s | 333.85 t/s | **+36.9 %** |
+| prefill, depth 131,072 | 237.24 t/s | 131.80 t/s | **+80.0 %** |
+| decode, depth 0 (mixed) | 16.08 t/s | 15.00 t/s | +7.2 % |
+| decode, depth 32,768 (mixed) | 14.41 t/s | 14.44 t/s | −0.2 % |
+| decode, depth 65,536 (mixed) | 10.87 t/s | 12.62 t/s | **−13.9 %** |
+| decode, depth 131,072 (mixed) | 8.87 t/s | 11.62 t/s | **−23.7 %** |
+| cold load to HTTP-ready | 8.17 s | 36.47 s | **4.5× faster** |
+| peak memory @ 262,144 ctx | 50.58 GiB | 48.60 GiB | +4 % |
+
+Read that honestly: **prefill, load time and short-context decode improve;
+deep-context decode on the 8-bit target gets worse.** The crossover on Q8 is
+around 32k of cached depth. frank-large runs one 262,144-token conversation,
+so a long agent session spends its life on the losing side of that line
+while a short turn wins. The Q4_K_XL target does not show the regression
+(gufo is +11.7 % to +17.0 % at every depth) — that is the honest reason to
+revisit the quantization separately from the runtime.
+
+## Build
+
+`scripts/build-gufo.sh` reproduces the runtime end to end. It clones the
+pinned tag, configures the `release` preset (HIP, `--offload-arch=gfx1151`),
+builds, installs, and finishes with `gufo diagnose`. Two host facts it
+encodes, both found the hard way on 2026-09-28:
+
+- **Ubuntu's `librocwmma-dev` must not be installed.** It ships a truncated
+  rocWMMA header set (the public headers without `rocwmma/internal/`) into
+  `/usr/include`, where it shadows AMD's complete copy under
+  `/opt/rocm/include`. The first build died on `'internal/accessors.hpp'
+  file not found` from `/usr/include/rocwmma/rocwmma.hpp`, and **a `-I`
+  cannot fix it**: in HIP mode the host compilation pass searches the system
+  directories first. The package has no reverse dependencies; the script
+  removes it.
+- **`--gcc-install-dir` must point at GCC 15.** Ubuntu 26.04 has
+  `/usr/include/c++/16`, and the ROCm clang picks the newest tree, whose
+  `<atomic_wait.h>` references `std::__detail::__notify_impl` at a symbol
+  version the installed `libstdc++.so.6` (GLIBCXX_3.4.35) does not export.
+  The link then fails with `undefined symbol`. Pinning the headers to the
+  GCC the runtime library came from fixes it.
+
+Verify an installed runtime with `bash scripts/build-gufo.sh --verify`, or
+`gufo diagnose` directly — it reports `[PASS]` plus the detected
+architecture, driver and queue budget.
+
+## Shape
+
+- **Model:** `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL` — 27B dense,
+  29.30 GiB. **Unchanged by the cutover**, still fetched and sha256-verified
+  by `scripts/download-frank-large.sh`.
+- **Context:** one session, full 262,144 tokens (the model's native window),
+  `SESSIONS=1`. The fleet's single stream is unchanged.
+- **Spec method:** DFlash2, gufo's **adaptive** block policy
+  (`--draft-policy adaptive`, `--draft-tokens 7`). The drafter is the same
+  z-lab `Qwen3.8-27B-DFlash2-Q4_K_M` (1.07 GiB).
+- **Sampling:** greedy (`--temperature 0`), as every prior frank-large
+  served. gufo's per-model recommendation for Qwen3.8 thinking is *sampled*
+  (1.0 / 0.95 / top-k 20) — that is a separate decision, deliberately not
+  bundled into a runtime swap.
+- **Memory:** gufo's own table puts the 8-bit target at **50.58 GiB** peak
+  with context capacity 262,144, against the iGPU's ~114 GiB GTT aperture.
+
+## Client contract — what actually changed
+
+Everything the fleet depended on still holds: port 7770, alias `frank-large`,
+`reasoning_content`, `tool_calls`, streaming SSE, and text-only (gufo
+auto-discovers a `mmproj` beside the model; this payload has none).
+
+The differences worth knowing:
+
+- **No custom chat template.** gufo compiles the official Qwen Jinja into the
+  binary and takes no `--chat-template-file`. The Qwen-Sharp v22.4.0 file
+  this endpoint used with llama.cpp is **no longer in the command line** —
+  the 2026-09-21 "73,600-token prompt" incident cannot recur, because there
+  is no fallback template to land in. Thinking defaults to on at `xhigh`
+  with prior reasoning preserved (`--think auto --preserve-thinking on`,
+  spelled out in the serve script).
+- **`reasoning_effort` accepts the full OpenAI set** (`none`, `minimal`,
+  `low`, `medium`, `high`, `xhigh`, `max`). Qwen-Sharp rejected anything
+  outside `{xhigh, medium, low}` with an HTTP 500, which is why `models.yml`
+  still carries a `reasoningEffortMap` fold for this model. **The fold is
+  kept on purpose**: removing it would silently move every `high` role from
+  `xhigh` to `high` effort in the same commit as the runtime swap. Drop it as
+  its own change once the effort curve is a decision rather than a side
+  effect.
+- **No KV-cache quantization flag.** gufo picks its own KV dtype; there is
+  no `--cache-type-k`. The old `CACHE_RAM=8192` tunable is gone with it.
+  Prompt reuse is on by default, and `--cache-disk` (restart-safe) is opt-in
+  and currently off.
+- **Health/readiness split:** `GET /health` is process liveness and is what
+  the deployer gates on; `GET /ready` is model/backend readiness.
+  `GET /metrics` is Prometheus-format.
+- **Long bodies:** `--max-request-bytes` defaults to 8 MiB and unknown
+  request fields are rejected rather than ignored.
+
+## Failure modes worth knowing
+
+- **A missing runtime is a deploy error, not a restart loop.** The handler
+  gates on `LLM_LARGE_RUNTIME` *before* it stops anything, and the serve
+  script exits with a `build-gufo.sh` pointer rather than exec'ing.
+- **Host memory is the ceiling, and it is shared.** frank-large and ComfyUI
+  both hold gfx1151 compute queues. gufo reads
+  `/sys/class/kfd/kfd/proc/*/queues/*/type` and caps itself
+  (`GPU_MAX_HW_QUEUES`, 2 resident for `serve llm`) so a full device
+  time-slices instead of failing; four HIP processes is the device ceiling.
+- **Load is seconds, not minutes** (8 s measured cold), so a redeploy no
+  longer spends 20 minutes in `LLM_WAIT`.
+- **A v0.1.x runtime is a moving target.** Rebuilds are not automatic —
+  `RUNTIME_DIR` names a version, and the deployer will not silently
+  substitute another.
+
+## Rollback
+
+[[frank-large-unsloth-qwen38-27b-q8xl]] — the unsloth b11030 prebuilt is
+still at `/home/dank/GitPackages/unsloth-llama-b11030/` and the payload is
+the same directory the current runtime reads. Roll back by reverting
+`RUNTIME_DIR` (and the gufo-only tunables) in `scripts/frank-large.env`,
+restoring the serve script's llama.cpp command, and redeploying.
+
+# Citations
+
+- Runtime: https://github.com/gufo-org/gufo (pinned `v0.1.1`, MIT)
+- Measured tables: `docs/models/qwen3.8-27b/BENCHMARKS.md` in that repo
+- API contract: `docs/SERVER.md`; flags: `docs/CLI.md`
+- Build: `scripts/build-gufo.sh`; serve: `scripts/serve-frank-large.sh`;
+  tunables: `scripts/frank-large.env`
+- Payload: `scripts/download-frank-large.sh`
+- Predecessor: [[frank-large-unsloth-qwen38-27b-q8xl]]