serverAppConfigs/serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md

---
type: reference
title: "gufo, Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)"
description: "The active frank-large runbook — gufo (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source on Dankclaw and serving the abliterated ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL with the z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU: the from-source build and its two host workarounds, the GGUF header evidence that made the payload swap safe, the one-session / full-window shape, why the fleet's Qwen-Sharp chat template CANNOT be used on this runtime and what that costs, the client-contract delta, the measured prefill/decode trade, and the failure modes worth knowing."
tags:
  - llm
  - gufo
  - swift-1.5
  - dankclaw
  - frank-large
  - strix-halo
  - gfx1151
  - q8
  - dflash2
  - rocm
timestamp: 2026-09-28
---

# frank-large — gufo, Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (DFlash2)

> **ACTIVE since 2026-09-28.** Replaces
> [[frank-large-unsloth-qwen38-27b-q8xl]] (stock llama.cpp, the unsloth
> b11030 prebuilt), which had replaced
> [[frank-large-kairic-edge-qwen38-27b]]. Same host, same port (7770), same
> alias (`frank-large`).
>
> **The payload also changed on 2026-09-28**, later the same day: the target
> went from `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL` to
> `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`'s
> `UD-Q8_K_XL` — an abliterated (uncensored) fine-tune of the same base. See
> [Payload](#payload) and, for what that costs, [the template
> law](#the-template-law-qwen-sharp-is-gone).

## Identity

- **Endpoint:** `frank-large` at `192.168.0.143:7770` (Dankclaw).
- **Runtime:** [gufo](https://github.com/gufo-org/gufo) — "the Strix Halo
  inference engine", a vertical engine built for exactly this hardware
  (Ryzen AI MAX+ 395 / Radeon 8060S, `gfx1151`, unified memory). Pinned at
  **v0.1.1** (commit `b0f8673`), built from source on the host to
  `/home/dank/GitPackages/gufo-v0.1.1/`.
- **Service shape:** root systemd unit `scripts/frank-large.service`
  (`User=dank`, `LimitMEMLOCK=infinity`), launched by
  `scripts/serve-frank-large.sh` with tunables from
  `scripts/frank-large.env`.

gufo ships **no prebuilt Linux binaries** (the v0.1.0/v0.1.1 release assets
are empty) and no Nix on this host, so the runtime is a source build. It is
also young: v0.1.0 landed 2026-09-28, the same day as this cutover.

## Why this engine

gufo's own measurement of the two engines on the same Qwen3.8-27B targets
(Strix Halo, `628ed18e` vs llama.cpp `b11069`, greedy, DFlash2):

| Q8_K_XL, DFlash2 | gufo | llama.cpp | delta |
|---|---:|---:|---:|
| prefill, depth 0 | 457.05 t/s | 333.85 t/s | **+36.9 %** |
| prefill, depth 131,072 | 237.24 t/s | 131.80 t/s | **+80.0 %** |
| decode, depth 0 (mixed) | 16.08 t/s | 15.00 t/s | +7.2 % |
| decode, depth 32,768 (mixed) | 14.41 t/s | 14.44 t/s | −0.2 % |
| decode, depth 65,536 (mixed) | 10.87 t/s | 12.62 t/s | **−13.9 %** |
| decode, depth 131,072 (mixed) | 8.87 t/s | 11.62 t/s | **−23.7 %** |
| cold load to HTTP-ready | 8.17 s | 36.47 s | **4.5× faster** |
| peak memory @ 262,144 ctx | 50.58 GiB | 48.60 GiB | +4 % |

Read that honestly: **prefill, load time and short-context decode improve;
deep-context decode on the 8-bit target gets worse.** The crossover on Q8 is
around 32k of cached depth. frank-large runs one 262,144-token conversation,
so a long agent session spends its life on the losing side of that line
while a short turn wins. The Q4_K_XL target does not show the regression
(gufo is +11.7 % to +17.0 % at every depth) — that is the honest reason to
revisit the quantization separately from the runtime.

## Build

`scripts/build-gufo.sh` reproduces the runtime end to end. It clones the
pinned tag, configures the `release` preset (HIP, `--offload-arch=gfx1151`),
builds, installs, and finishes with `gufo diagnose`. Two host facts it
encodes, both found the hard way on 2026-09-28:

- **Ubuntu's `librocwmma-dev` must not be installed.** It ships a truncated
  rocWMMA header set (the public headers without `rocwmma/internal/`) into
  `/usr/include`, where it shadows AMD's complete copy under
  `/opt/rocm/include`. The first build died on `'internal/accessors.hpp'
  file not found` from `/usr/include/rocwmma/rocwmma.hpp`, and **a `-I`
  cannot fix it**: in HIP mode the host compilation pass searches the system
  directories first. The package has no reverse dependencies; the script
  removes it.
- **`--gcc-install-dir` must point at GCC 15.** Ubuntu 26.04 has
  `/usr/include/c++/16`, and the ROCm clang picks the newest tree, whose
  `<atomic_wait.h>` references `std::__detail::__notify_impl` at a symbol
  version the installed `libstdc++.so.6` (GLIBCXX_3.4.35) does not export.
  The link then fails with `undefined symbol`. Pinning the headers to the
  GCC the runtime library came from fixes it.

Verify an installed runtime with `bash scripts/build-gufo.sh --verify`, or
`gufo diagnose` directly — it reports `[PASS]` plus the detected
architecture, driver and queue budget.

## Shape

### Payload

- **Model:** `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`,
  file `UD-Q8_K_XL` — 27B dense, 31,457,990,272 bytes, fetched and
  sha256-verified (`9076f99a…`, the Hub LFS oid) by
  `scripts/download-frank-large-swift.sh`. It is an **abliterated
  (uncensored)** fine-tune of Qwen3.8-27B. The repository also publishes an
  `mmproj-BF16.gguf`, which is **not** used: this endpoint is text-only.
- **Licence:** the repo declares `other` — the **SWIFT OPEN LICENSE 1.0**,
  which is *not* an OSI licence. That is a real constraint on redistribution,
  noted here so nobody discovers it later. It is fine for a private fleet.
- **Replaced:** `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL`, still on disk and
  still the rollback target for the payload.

The two targets are **structurally interchangeable for this engine**, which
is why the swap needed no engine change. Header-diffed directly:

| | unsloth | Swift-1.5 |
|---|---|---|
| `general.architecture` | `qwen35` | `qwen35` |
| tensors (count, names, order) | 866 | 866, **identical** |
| per-tensor `ggml_type` | Q8_0×453, F32×360, BF16×53 | **identical** |
| per-tensor dims | — | **0 mismatches** |
| MTP head | 4 tensors at `blk.64.nextn.*` | identical |
| geometry | 65 blk, 5120 embd, 17408 FFN, 24/4 heads | identical |

The two metadata differences that look alarming are both no-ops, and both
were confirmed against gufo's source rather than assumed:

- **`general.file_type` is 15 vs 7.** gufo's `gguf_reader.cpp` states it is
  *"deliberately not consulted… The Unsloth UD-\*_K_XL artifacts this repo
  targets carry a stale one — UD-Q8_K_XL records ftype 15 (Q4_K_M) … The
  tensor table always is."* The file gufo already served lies about itself
  here, so gufo reads the tensor table.
- **Swift adds `qwen35.attention.recurrent_layers`.** gufo never reads that
  key anywhere in `src/`; layer kinds come from `full_attention_interval`,
  which both files carry as `4` and which agrees with it.

### Serving shape

- **Context:** one session, full 262,144 tokens (the model's native window),
  `SESSIONS=1`. The fleet's single stream is unchanged.
- **Spec method:** DFlash2, gufo's **adaptive** block policy
  (`--draft-policy adaptive`, `--draft-tokens 7`). The drafter is the
  **unchanged** z-lab `Qwen3.8-27B-DFlash2-Q4_K_M` (1.07 GiB) — it did not
  move with the target, which is why `DRAFTER_DIR` is a separate tunable
  from `MODEL_DIR` and why the deployer's stray-run sweep keys on the
  drafter's filename rather than the target's.
- **Sampling:** greedy (`--temperature 0`) server-side, as every prior
  frank-large served. The harness overrides per request via `extraBody`
  (1.0 / 0.95 / top-k 20).
- **Memory:** gufo's own table puts the 8-bit target at **50.58 GiB** peak
  with context capacity 262,144, against the iGPU's ~114 GiB GTT aperture.

### Measured at the 2026-09-28 cutover

Live on Dankclaw. The **target** numbers are the Swift-1.5 load test
(a standalone run under the production flags, service stopped first); the
**engine** numbers are from the gufo cutover earlier the same day and are
unchanged by the payload swap.

- **9.2 s** `load_started` → `load_completed` for the Swift target, against a
  20-minute `LLM_WAIT` budget. The unsloth target loaded in 13 s.
- **66.5 GiB** of the 116.7 GiB iGPU device at context capacity 262,144;
  host RSS 30.6 GiB. No loader warnings, no skipped sections.
- Contract battery, all green on the Swift target: `/v1/models` →
  `context_length 262144`, a chat completion carrying both `content` and
  `reasoning_content`, a `tool_calls` response with
  `finish_reason: tool_calls`, SSE streaming, and a **two-turn tool loop**
  (tool call → tool result → follow-up) with prior reasoning preserved.
  Arithmetic spot-check: "17*23" → `391`.
- Behavioural spot-check that this is the abliterated model: asked for a
  two-sentence lockpicking scene, it wrote one without hedging or refusing.
- From the gufo cutover, on the unsloth target: a short-prompt live request
  returned
  `Server-Timing: ttft;dur=549.8, inter_token;dur=24.2` (≈41 tok/s).
  Short-prompt decode is gufo's *winning* regime; see the depth table above
  before quoting it as the endpoint's number.

### Tuning: the served flags are already optimal (2026-09-28)

Frank asked for iterative decode/prefill improvement at a 32k working point
without touching the model or the quantization. **Result: no win exists at
this engine's served settings, and nothing was changed.** Measured on real
repo prose and shell sources at **32,762 prompt tokens**, salted per run so
a warm prefix cache could not help, 3–5 repetitions each.

| lever | range swept | outcome |
|---|---|---|
| `--prefill-chunk` | 512 → 8192 | **no effect** — 410.7–412.4 t/s prefill across a 16× change |
| `--draft-tokens` | 5 → 10 | **flat** — 13.79–15.86 t/s decode, inside run-to-run noise |
| `--draft-policy` | fixed vs adaptive | 35.90 vs 35.50 (bench) — no effect |
| `--min-draft-tokens` | — | **rejected by the engine**: "DFlash2 requires … 1" |
| MTP backend | — | **unsupported for Q8**: `token_embd.weight` is Q8_0 |
| `--context` 262144 → 40960 | diagnostic | 13.93 vs 14.71 t/s — no gain, and it would break the declared client context |

The confirmation round at 5 reps **reversed** the 3-rep leaders: the pair
that looked best (`dt9 + chunk4096`) came last at 12.86 t/s, and the
**deployed configuration came first at 14.71 t/s**. Every difference sits
inside the noise band.

What *does* move the number is speculative decoding itself: **6.65 t/s off →
~15 t/s on, a 5.3× step** — and that is already switched on. Prefill is
insensitive to everything tried, which is consistent with gufo's own
`EXPERIMENTS.md`, where dozens of kernel-level alternatives are recorded as
rejected for "no release throughput improvement".

**Do not repeat this investigation without new information.** Two traps
cost most of the time, and both are in AGENTS.md's benchmarking policy:
`gufo bench` reports `acceptance=0.933 ± 0.000` on an identical trace hash
and **inverts** the draft-width ranking; and a live A/B needs ≥128 generated
tokens or `inter_token` is first-token noise.

## The template law: Qwen-Sharp is gone

`scripts/qwen-sharp-v22.4.0.jinja` is the fleet's chat template — it is
wired into `frank-primary` as `R9V_CHAT_TEMPLATE_FILE` and it was
`frank-large`'s `--chat-template-file` under llama.cpp. **frank-large can no
longer use it**, and the reason is stronger than "gufo lacks the flag":

> gufo has **no Jinja engine at all.** `src/models/qwen/chat_template.cpp`
> is 587 lines of hand-written C++ that appends strings to build ChatML.
> There is no `--chat-template-file`, no environment override, and no
> template body anywhere in the binary to point at. Qwen-Sharp is 453 lines
> of Jinja2 using `namespace()`, macros, `raise_exception` and image/video
> branches — supplying it would mean **reimplementing it in C++**, not
> handing gufo a file.

Given the choice on 2026-09-28, the decision was to **ship on gufo's stock
Jinja and accept the loss**. What that gives up, concretely:

| Qwen-Sharp feature | On gufo |
|---|---|
| `terse` system-prompt injection (the headline behaviour) | **gone** |
| in-band `<\|think_off\|>` / `<\|think_xhigh\|>` control | **gone** |
| `tool_call_format` `json` \| `xml` switch | **gone** |
| `max_tool_arg_chars` / `max_tool_response_chars` truncation | **gone** |
| `add_vision_id` → `Picture N:` prefixes | **gone** |
| effort instruction strings in the system prompt | kept (gufo has its own, same text) |
| `preserve_thinking` | kept (`--preserve-thinking on`) |

The `terse` injection is the one that matters most here, and it is worth
being blunt about the combination: this endpoint now runs an **abliterated**
model *without* the prompt that tells it to stop waffling. If verbosity
shows up, the cheapest fix is to send the `terse` text as an ordinary
**system message from the client** (`models.yml` / the harness) rather than
through a template — that recovers the behaviour without forking the
renderer. The full-fidelity options are reverting to llama.cpp or porting
Qwen-Sharp into the C++ renderer; both are worse trades today, and the
second is the same class of change that produced the 2026-09-21
"73,600-token prompt" incident.

## Client contract — what actually changed

Everything the fleet depended on still holds: port 7770, alias `frank-large`,
`reasoning_content`, `tool_calls`, streaming SSE, and text-only (gufo
auto-discovers a `mmproj` beside the model; this payload has none).

The differences worth knowing:

- **No custom chat template — see
  [The template law](#the-template-law-qwen-sharp-is-gone) above**, which is
  the most consequential change on this endpoint. Thinking defaults to on at
  `xhigh` with prior reasoning preserved (`--think auto
  --preserve-thinking on`, spelled out in the serve script).
- **`reasoning_effort` accepts the full OpenAI set** (`none`, `minimal`,
  `low`, `medium`, `high`, `xhigh`, `max`), all verified 200 against the
  live endpoint. Qwen-Sharp rejected anything outside `{xhigh, medium, low}`
  with an HTTP 500, which is why `models.yml` still carries a
  `reasoningEffortMap` fold for this model. **The fold is kept on
  purpose**: removing it would silently move every `high` role from `xhigh`
  to `high` effort. Drop it as its own change.
- **Every field the harness sends is accepted.** `top_k`, `min_p`,
  `presence_penalty`, `repetition_penalty` and `preserve_thinking` are all
  outside gufo's documented request subset, and all returned 200. They are
  applied from `models.yml`'s `extraBody` unchanged.
- **No KV-cache quantization flag.** gufo picks its own KV dtype; there is
  no `--cache-type-k`. The old `CACHE_RAM=8192` tunable is gone with it.
  Prompt reuse is on by default, and `--cache-disk` (restart-safe) is opt-in
  and currently off.
- **Health/readiness split:** `GET /health` is process liveness and is what
  the deployer gates on; `GET /ready` is model/backend readiness.
  `GET /metrics` is Prometheus-format.

- **Long bodies:** `--max-request-bytes` defaults to 8 MiB.

## Failure modes worth knowing

- **A missing runtime is a deploy error, not a restart loop.** The handler
  gates on `LLM_LARGE_RUNTIME` *before* it stops anything, and the serve
  script exits with a `build-gufo.sh` pointer rather than exec'ing.
- **Host memory is the ceiling, and it is shared.** frank-large and ComfyUI
  both hold gfx1151 compute queues. gufo reads
  `/sys/class/kfd/kfd/proc/*/queues/*/type` and caps itself
  (`GPU_MAX_HW_QUEUES`, 2 resident for `serve llm`) so a full device
  time-slices instead of failing; four HIP processes is the device ceiling.
- **Load is seconds, not minutes** (9.2 s measured cold on the Swift
  target), so a redeploy no longer spends 20 minutes in `LLM_WAIT`.
- **A payload is two files in two directories.** The target and the drafter
  no longer live together, so "the payload is present" is not one check; the
  deployer gates on both paths and the serve script refuses to start if
  either is absent. A symlinked "one payload directory" would hide that.
- **A v0.1.x runtime is a moving target.** Rebuilds are not automatic —
  `RUNTIME_DIR` names a version, and the deployer will not silently
  substitute another.

## Rollback

Two independent rollbacks, because two things changed on 2026-09-28:

1. **Payload only** — repoint `MODEL_FILE` in `scripts/frank-large.env` at
   `/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-UD-Q8_K_XL.gguf`
   and redeploy. **The 29.3 GiB target was reclaimed on 2026-09-28**, so this
   path now needs a re-fetch first: `scripts/download-frank-large.sh`, then
   the env edit. The DFlash2 drafter was never touched and still lives in
   that directory.
2. **Engine** — [[frank-large-unsloth-qwen38-27b-q8xl]]: the unsloth b11030
   prebuilt is still at `/home/dank/GitPackages/unsloth-llama-b11030/`.
   Revert `RUNTIME_DIR` and the gufo-only tunables, restore the serve
   script's llama.cpp command, and redeploy. This is also the only way to
   get `qwen-sharp-v22.4.0.jinja` back.

# Citations

- Runtime: https://github.com/gufo-org/gufo (pinned `v0.1.1`, MIT)
- Measured tables: `docs/models/qwen3.8-27b/BENCHMARKS.md` in that repo
- API contract: `docs/SERVER.md`; flags: `docs/CLI.md`; the C++ renderer
  that rules out a Jinja template:
  `src/models/qwen/chat_template.cpp` (587 lines)
- GGUF header evidence for the payload swap: both files' headers were parsed
  and diffed directly (architecture, all 866 tensor names in order, per-tensor
  `ggml_type` and dims) before either was loaded
- Build: `scripts/build-gufo.sh`; serve: `scripts/serve-frank-large.sh`;
  tunables: `scripts/frank-large.env`
- Payload: `scripts/download-frank-large-swift.sh` (target),
  `scripts/download-frank-large.sh` (DFlash2 drafter)
- Predecessor: [[frank-large-unsloth-qwen38-27b-q8xl]]