docs: GLM-5.3-Flash readiness revised — tcclaviger fork has native MXFP4/DSFP4

Source reading in the qwen-fp8 container: glm5next is registered with an amd/ backend dir + DSA indexer; r4d.so has native moe_dsfp4/moe_dsfp4_mt1 entries (has_dsfp4()==True), so the old 'MXFP4 = emulation, missing kernels' verdict is dead for this fork. DSFP4 = MXFP4 weights + DeepSeek activation law. Offload recipes exist for mxfp4 and W4A16 (R4dW4a16MoEExperts in offloader/expert.py:114). GLM gate is now the payload format (compressed-tensors MXFP4/W4A16, none public) + a HIP boot probe.
docs: final 256k verdict — boots at 4.03-4.5 GiB KV, band 34.1-36.5 straddles target
2026-09-29 23:28:04 -07:00 · 2026-09-29 21:55:58 -07:00 · 2026-09-29 21:22:21 -07:00 · 2026-09-29 05:06:06 -07:00 · 2026-09-29 00:00:49 -07:00 · 2026-09-28 23:29:05 -07:00
22 changed files with 2034 additions and 278 deletions
@@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
 | Dankserver | `frank-primary` | R9V radiance-vLLM fork `drwolfen/radiance-vllm-qwen4exp` (docker-pinned image bundle `2dac17a2`, docker→podman shim, root systemd unit) | unsloth Qwen3.8-Flash-Next-GGUF Q8_0 (175.30 GiB, 6 GGUF shards) with the blockfp8 MTP4 head (depth 4) and mmproj, TP=2 tiered experts (80-slot LRU cache; PLE n-gram PINNED in host RAM 54.4 GB; quantized inject-shard dev-overlay adapter), CED off, 131072 ctx single stream, Qwen-Sharp template, tool parser `qwen3_coder`, reasoning channel `reasoning` | 7770 |
-| Dankclaw | `frank-large` | unsloth llama.cpp b11030 prebuilt (ROCm gfx1151, root systemd unit) | unsloth Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, 31.5 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) on the gfx1151 iGPU (Strix Halo LPDDR, 114 GiB GTT), 262144 ctx single stream (1 slot), q8_0 KV, text only | 7770 |
+| Dankclaw | `frank-large` | gufo `gufo-org/gufo` v0.1.1, the Strix Halo engine, BUILT FROM SOURCE (root systemd unit; links the host ROCm 7.2.4) | Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (27B dense, ABLITERATED/uncensored, 29.3 GiB, SWIFT OPEN LICENSE 1.0) with the z-lab DFlash2 Q4_K_M drafter (adaptive block policy, cap 7) on the gfx1151 iGPU (Strix Halo LPDDR, 114 GiB GTT), 262144 ctx single session, greedy, gufo's compiled-in Qwen renderer (NO Qwen-Sharp), text only | 7770 |
 | Dankclaw | `frank-small` | mainline llama.cpp master 37b53fd (root systemd unit) | HauhauCS Gemma4-12B-QAT-Uncensored-Balanced Q4_K_M (BF16 mmproj + MTP assistant sidecar, `draft-mtp,ngram-mod`) on the RTX 3090, 1048576 pool / 4 slots of 262,144, q8_0 KV, the fleet's fastest vision endpoint | 7771 |

 Retired endpoints (`frank-npu`, and every prior runtime) are in
@@ -208,13 +208,27 @@ archive.

 **STOP the production service BEFORE benchmarking** — these servers serve
 real users, and a bench beside a live service starves the GPU and corrupts
-results. Use the runtime's own harness: `llama-bench` for every llama.cpp
-endpoint (all three are llama.cpp now). Rolling your own bench
+results. Use the runtime's own harness: `llama-bench` for the llama.cpp
+endpoints, `gufo bench` for frank-large. Rolling your own bench
 requires asking first.
 Scratch ports are not used on LLM hosts — a
 test process cannot share the single GPU anyway; validate against the service
 itself.

+**`gufo bench` cannot rank speculative-decode settings, and will invert the
+ranking.** Measured 2026-09-28 on frank-large at depth 32768: its corpus
+reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
+draft widths, i.e. the same highly-predictable text every run, and 93 % of
+every proposal accepted. It reported 29–37 t/s decode where real traffic
+gives ~15 t/s, and it made `--draft-tokens 16` look 6 % faster than 7 when
+real text made it 14 % *slower*. Use it for shape, not for A/B. For an A/B,
+replay **real** text through the live server and read the `Server-Timing`
+header — but note the unit rides in the **name** half (`ttft;dur=402.485`),
+so splitting on `=` alone keys the dict `ttft;dur` and every lookup silently
+returns nothing. Generate **at least 128 tokens** for the measurement: at 8
+tokens, `inter_token` is first-token noise and the run-to-run spread swamps
+every real difference.
+
 This covers **any GPU experiment, not just benchmarks** — including a bare
 2-rank RCCL/`torch.distributed` probe. On 2026-09-16 one such probe was run
 against the live service "because it only needs 200 MB of VRAM"; the host went
@@ -227,7 +241,7 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
 merely wedged kernel usually still answers ARP.

 1. Stop and mask (self-healing would restart it):
-   - Dankclaw `frank-large` (root unit, stock llama.cpp on the iGPU):
+   - Dankclaw `frank-large` (root unit, gufo on the iGPU):
     `sudo systemctl stop frank-large && sudo systemctl mask frank-large`
   - Dankclaw `frank-small` (root unit, llama.cpp on the 3090):
     `sudo systemctl stop frank-small && sudo systemctl mask frank-small`
@@ -277,47 +291,111 @@ FLM NPU endpoint on 2026-09-11 and returned on 2026-09-17 with the Gemma4
 projector's audio encoder (frank-small transcribes WAV verbatim even though
 `audio` is omitted from its `input` list — OMP's schema rejects it).

-### frank-large — stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (Dankclaw, ACTIVE)
+### frank-large — gufo, Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (Dankclaw, ACTIVE)

-The unsloth llama.cpp prebuilt `app-b11030-mix-5ff778e` (ROCm gfx1151,
-`/home/dank/GitPackages/unsloth-llama-b11030/`, bundles its own ROCm
-runtime — no host SDK gate) as a **root systemd unit**
+**gufo** (`gufo-org/gufo`) v0.1.1 — "the Strix Halo inference engine", a
+vertical engine built for exactly this hardware (Ryzen AI MAX+ 395 /
+Radeon 8060S, `gfx1151`) — as a **root systemd unit**
 (`scripts/frank-large.service`, `User=dank`), serving
-`unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL` (27B dense, 31.5 GiB,
-sha256-verified) on port 7770 from the gfx1151 iGPU. One stream, full
-262,144 ctx (the model's native window, `PARALLEL_SLOTS=1`), DFlash2 spec
-decode (the z-lab `Qwen3.8-27B-DFlash2-Q4_K_M` drafter, depth 4,
-`SPEC_DRAFT_N_MAX=4`), reasoning extracted to `reasoning_content`, and
-**no vision projector**. It replaced the Kairic Edge ROCmFPX build (jcbtc
-Qwen3.8-27B-IU4-Kairic-Edge), which had replaced the unsloth
-Qwen3.8-Flash-Next UD-IQ4_XS.
+`ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`'s `UD-Q8_K_XL`
+(27B dense, 29.3 GiB, sha256-verified; **abliterated/uncensored**, and under
+the *non-OSI* SWIFT OPEN LICENSE 1.0) on port 7770 from the gfx1151 iGPU.
+One session, full 262,144 ctx (the model's native window, `SESSIONS=1`),
+DFlash2 spec decode (the z-lab `Qwen3.8-27B-DFlash2-Q4_K_M` drafter, gufo's
+**adaptive** block policy, `--draft-tokens 7`), greedy sampling, reasoning
+extracted to `reasoning_content`, and **no vision projector**.

-The spec method is DFlash2 (block-diffusion drafter) at depth 4. The
-drafter is loaded via `--model-draft`; the chain is `--spec-type
-draft-dflash` with `--spec-draft-device ROCm0`. KV is **q8_0** (the
-unsloth b11030 build does not support fp8 KV; q8_0 is the
-highest-precision 8-bit option, ~8 GiB at 262k). The serve script
-(`scripts/serve-frank-large.sh`) validates binary → main GGUF → drafter
-and exits with a pointer message if any is missing, then `exec`s
-`llama-server` with `--jinja --alias frank-large --chat-template-kwargs
-'{"preserve_thinking":true}' --chat-template-file $TEMPLATE_FILE` (the
-Qwen-Sharp v22.4.0 template; without `--chat-template-file` the binary
-falls back to the built-in `peg-native` template, which produces a
-~73,600-token prompt for a ~250-token request).
+**Two swaps landed on 2026-09-28**, both same day: the engine (from the
+unsloth llama.cpp b11030 prebuilt, which had replaced Kairic Edge ROCmFPX)
+and then the payload (from `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL`, still on
+disk and the payload-only rollback). The two targets are structurally
+interchangeable for gufo — same `qwen35` architecture, the same 866 tensors
+in the same order, the same per-tensor Q8_0/F32/BF16 layout, the same MTP
+head at `blk.64` — verified by parsing and diffing both GGUF headers before
+either was loaded. **The DFlash2 drafter did not move**, so it has its own
+`DRAFTER_DIR` and the deployer's stray-run sweep keys on the drafter's
+filename, not the target's.

-The iGPU addresses **114 GiB** (the GTT aperture over the shared LPDDR,
-`gtt_total` in `/sys/class/drm/card1/device/`; `rocm-smi`'s 2 GiB "dedicated"
-is not the budget). The 27B dense model (31.5 GiB) is mmap'd in with
-`--n-gpu-layers 999`; budget at 262k ≈ 41-43 GiB (weights + KV + buffers),
-leaving ~71 GiB of aperture headroom. The 8 GiB prompt-cache budget
-(`CACHE_RAM=8192`) is the qualified config.
+**The runtime is built from source on the host** —
+`/home/dank/GitPackages/gufo-v0.1.1/`, reproducible with
+`scripts/build-gufo.sh` (`--verify` re-checks an installed tree). gufo ships
+no prebuilt Linux binaries and the v0.1.0/v0.1.1 release assets are empty.
+Unlike the retired llama.cpp prebuilt, which bundled its own ROCm, **gufo
+links the host ROCm 7.2.4 through its RPATH**, so the HIP dev packages must
+stay installed. Two host workarounds the build script encodes:

-Fetch: `scripts/download-frank-large.sh` (per-file, each file verified
-against its declared byte count + sha256; `--verify-only` re-checks).
+- **Ubuntu's `librocwmma-dev` must be absent.** It installs a truncated
+  rocWMMA header set into `/usr/include` that shadows AMD's complete one
+  and breaks the build with `'internal/accessors.hpp' file not found`. A
+  `-I` cannot fix it — in HIP mode the host pass searches the system
+  directories first. The package has no reverse dependencies.
+- **`--gcc-install-dir` must name GCC 15.** Ubuntu 26.04 ships
+  `/usr/include/c++/16`, which the ROCm clang prefers; its `<atomic_wait.h>`
+  needs a `std::__detail::__notify_impl` symbol version the installed
+  `libstdc++.so.6` (GLIBCXX_3.4.35) does not export, and the link fails.

-Runbook: `serverAppConfigs-docs/LLM/frank-large-unsloth-qwen38-27b-q8xl.md`.
-Rollback: [[frank-large-kairic-edge-qwen38-27b]] (payload and runtime
-intact on disk).
+`gufo diagnose` is the preflight and must report `[PASS]` with `gfx1151`.
+It also self-caps compute queues (`GPU_MAX_HW_QUEUES`, 2 resident for
+`serve llm`) after reading `/sys/class/kfd/kfd/proc/*/queues/*/type`, which
+matters because ComfyUI holds queues on the same device; four HIP processes
+is the device ceiling.
+
+**The client contract changed, and the chat template is the sharp edge.**
+**gufo has no Jinja engine at all** — `src/models/qwen/chat_template.cpp` is
+587 lines of hand-written C++ that appends strings, with no
+`--chat-template-file`, no env override, and no template body to point at.
+`scripts/qwen-sharp-v22.4.0.jinja` therefore **cannot** be supplied to
+frank-large: that is a port, not a config line. Given the choice on
+2026-09-28 the decision was to ship on gufo's stock renderer and accept the
+loss — the `terse` system-prompt injection, the in-band `<|think_*|>`
+control, the `json|xml` `tool_call_format` switch, the tool-argument
+truncation and `add_vision_id` prefixes are all gone. gufo's own renderer
+does keep the effort instruction strings and `preserve_thinking`, which are
+what the harness depends on. **This is the thing to watch**: the endpoint
+now runs an *abliterated* model without the prompt that tells it to stop
+waffling. The cheap fix, if verbosity shows up, is to send the `terse` text
+as an ordinary system message from `models.yml` — not a template change.
+`reasoning_effort` accepts the full OpenAI set (all six levels verified
+200), so `models.yml`'s `reasoningEffortMap` fold is no longer *required* —
+it is kept deliberately, so that removing it does not silently move every
+`high` role from `xhigh` to `high` effort inside a swap. And there is no
+KV-quantization or prompt-cache-RAM flag; prompt reuse is on by default and
+`--cache-disk` is opt-in and off.
+
+**Read the perf trade before assuming a win.** gufo's own Q8_K_XL tables
+(vs llama.cpp b11069, DFlash2) show prefill +37 % to +80 %, cold load 4.5×
+faster, decode +7 % at zero depth and +40 % at 4 users — but **−14 % at
+65,536 tokens of depth and −24 % at 131,072**. On the 8-bit target the
+crossover is ~32k of cached context, and frank-large runs one 262,144-token
+conversation, so long agent sessions sit on the losing side of that line.
+The Q4_K_XL target shows no such regression; revisiting the quantization is
+a separate decision from the runtime. Peak memory is 50.58 GiB at the full
+262,144 context against the iGPU's 114 GiB GTT aperture.
+
+The serve script (`scripts/serve-frank-large.sh`) validates runtime → target
+GGUF → drafter and exits with a pointer message if any is missing, then
+`exec`s `gufo serve --host 0.0.0.0 --port 7770 --sessions 1 llm --model
+…--served-model-name frank-large --context 262144 --speculative dflash2 …`.
+The deployer's `deploy_llm_large` gates on the **runtime binary** as well
+as both payload paths, and returns before it stops anything.
+
+Fetch: `scripts/download-frank-large-swift.sh` (the target; per-file byte
+count + sha256 against the Hub LFS oid, no auth header — the repo is public
+and a token on the curl command line is world-readable) and
+`scripts/download-frank-large.sh` (the DFlash2 drafter, which did not move).
+Both take `--verify-only`.
+
+Runbook: `serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md`.
+Rollback, two ways: **payload only** — re-fetch with
+`scripts/download-frank-large.sh` and repoint `MODEL_FILE` at
+`/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-UD-Q8_K_XL.gguf`; that
+29.3 GiB target was **reclaimed on 2026-09-28** (staged to
+`~/.AGENTTRASH/20260928-retired-models/`, restorable from there), so it is
+no longer a one-minute flip. **Engine** —
+[[frank-large-unsloth-qwen38-27b-q8xl]], the b11030 prebuilt at
+`/home/dank/GitPackages/unsloth-llama-b11030/`; the only way to get
+Qwen-Sharp back. The DFlash2 drafter was never reclaimed and still lives in
+`Qwen3.8-27B-UD-Q8_K_XL/`.

 ### frank-small — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)

@@ -771,7 +849,7 @@ Active:

 | Service | Runbook |
 |---------|---------|
-| frank-large (stock llama.cpp Qwen3.8-27B UD-Q8_K_XL, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-unsloth-qwen38-27b-q8xl.md` |
+| frank-large (gufo Qwen3.8-27B UD-Q8_K_XL, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md` |
 | frank-small (llama.cpp Gemma4-12B QAT, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-gemma4-12b.md` |
 | frank-primary (R9V radiance-vLLM Qwen3.8-Flash-Next Q8_0, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-r9v-radiance-qwen38-q8.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
@@ -781,6 +859,7 @@ Retired (rollback targets and history):

 | Runtime | Host | Window | Runbook |
 |---------|------|--------|---------|
+| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-21 → 09-28 | `frank-large-unsloth-qwen38-27b-q8xl.md` |
 | R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-primary`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-primary-r9v-radiance-qwen38-uncensored.md` |
 | llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-primary`) | Dankserver | 2026-09-26 → 09-27 | `frank-primary-llamacpp-qwen38-flash-next-q8.md` |
 | ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-large`) | Dankclaw | 2026-09-15 → 09-16 | `frank-large-ik-uncensored.md` |
@@ -13,7 +13,7 @@ one deploy script.
 | Host | User | IP | Podman stack | LLM service | LLM port |
 |------|------|----|--------------|-------------|----------|
 | dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
-| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 |
+| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |

 ## Active Services

@@ -0,0 +1,96 @@
+#!/usr/bin/env bash
+# Build the gufo runtime that serves frank-large on Dankclaw.
+#
+#   bash scripts/build-gufo.sh              # build + install the pinned tag
+#   bash scripts/build-gufo.sh --verify     # only check the installed tree
+#
+# gufo (gufo-org/gufo) is the Strix Halo inference engine: it targets
+# gfx1151 specifically and has no prebuilt Linux release binaries, so the
+# runtime is built from the pinned git tag and installed to
+# /home/dank/GitPackages/gufo-<tag>/. Everything below is what Dankclaw
+# needs on top of AMD's Linux instructions.
+#
+# The host already carries the ROCm 7.2.4 stack (/opt/rocm -> /etc/
+# alternatives/rocm) with the HIP dev packages, so this script installs
+# only the ordinary system libraries gufo links against.
+#
+# Two host facts this build depends on, both discovered on 2026-09-28:
+#
+#   1. Ubuntu's librocwmma-dev ships a TRUNCATED rocWMMA header set (the
+#      public headers without rocwmma/internal/) into /usr/include, where
+#      it shadows AMD's complete copy under /opt/rocm/include. The first
+#      build died on "'internal/accessors.hpp' file not found" from
+#      /usr/include/rocwmma/rocwmma.hpp. A -I cannot fix it: in HIP mode
+#      the host compilation pass searches the system directories first.
+#      The package has no reverse dependencies, so it is removed.
+#   2. The ROCm clang picks the NEWEST /usr/include/c++ tree, which on
+#      Ubuntu 26.04 is GCC 16's. Its <atomic_wait.h> emits a symbol
+#      reference the installed libstdc++ (GLIBCXX_3.4.35, GCC 15) does not
+#      export, and the link fails with "undefined symbol:
+#      std::__detail::__notify_impl". --gcc-install-dir pins the headers to
+#      the same GCC the runtime library came from.
+set -euo pipefail
+
+GUFO_TAG="${GUFO_TAG:-v0.1.1}"
+GUFO_REPO="${GUFO_REPO:-https://github.com/gufo-org/gufo}"
+# The commit the tag resolved to when this runtime was qualified. A tag that
+# moves is an unreproducible runtime, so the pin is checked, not assumed.
+GUFO_COMMIT="${GUFO_COMMIT:-b0f8673470a3dcc2b5e9c9546c48a40b82b5ff31}"
+SRC_DIR="${GUFO_SRC_DIR:-/home/dank/GitPackages/gufo-src}"
+PREFIX="${GUFO_PREFIX:-/home/dank/GitPackages/gufo-${GUFO_TAG}}"
+JOBS="${JOBS:-$(nproc)}"
+# The GCC toolchain that owns the installed libstdc++ on this host.
+GCC_DIR="${GCC_DIR:-/usr/lib/gcc/x86_64-linux-gnu/15}"
+
+# check_pin — refuse a source tree that is not the qualified commit.
+check_pin() {
+    local commit
+    commit="$(git -C "$SRC_DIR" rev-parse HEAD 2>/dev/null || true)"
+    if [[ -n "$commit" && "$commit" != "$GUFO_COMMIT" ]]; then
+        echo "GUFO_COMMIT mismatch: $SRC_DIR is at $commit, pinned $GUFO_COMMIT" >&2
+        exit 1
+    fi
+}
+
+if [[ "${1:-}" == "--verify" ]]; then
+    # The pin first: a moved tag is the more urgent thing to hear about, and
+    # it is the check that must not be masked by a missing install.
+    check_pin
+    [[ -x "$PREFIX/bin/gufo" ]] || { echo "no gufo at $PREFIX/bin/gufo" >&2; exit 1; }
+    "$PREFIX/bin/gufo" diagnose | tail -n 5
+    exit 0
+fi
+
+echo "== system libraries gufo links against =="
+sudo -n apt-get install -y -qq --no-install-recommends \
+    build-essential cmake ninja-build pkg-config \
+    libicu-dev libcurl4-openssl-dev libssl-dev libpng-dev libjpeg-dev ffmpeg
+
+echo "== dropping Ubuntu's truncated rocWMMA headers =="
+if dpkg -s librocwmma-dev >/dev/null 2>&1; then
+    sudo -n apt-get remove -y -qq librocwmma-dev
+fi
+
+echo "== source: $GUFO_REPO @ $GUFO_TAG =="
+if [[ ! -d "$SRC_DIR/.git" ]]; then
+    mkdir -p "$(dirname "$SRC_DIR")"
+    git clone --depth 1 --branch "$GUFO_TAG" "$GUFO_REPO" "$SRC_DIR"
+fi
+git -C "$SRC_DIR" fetch --depth 1 origin "refs/tags/$GUFO_TAG"
+git -C "$SRC_DIR" checkout -q --detach "refs/tags/$GUFO_TAG"
+echo "   $(git -C "$SRC_DIR" rev-parse HEAD)"
+check_pin
+
+echo "== configure + build (release, HIP gfx1151) =="
+cd "$SRC_DIR"
+toolchain="--gcc-install-dir=$GCC_DIR"
+cmake --preset release \
+    -DCMAKE_INSTALL_PREFIX="$PREFIX" \
+    -DCMAKE_HIP_FLAGS="$toolchain" \
+    -DCMAKE_EXE_LINKER_FLAGS="$toolchain"
+cmake --build --preset release --parallel "$JOBS"
+cmake --install build/release
+
+echo "== verify =="
+"$PREFIX/bin/gufo" diagnose
+echo "frank-large runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+# Single source of truth for Dankserver GLM-5.3 state.
+#
+# This exists because the ad-hoc status commands kept producing WRONG readings.
+# The specific trap: /proc/meminfo separates key from value with a TAB, so the
+# common idiom
+#   grep MemAvailable /proc/meminfo | tr -s " " | cut -d" " -f2
+# yields an empty field (tab is not a space) and awk then prints 0.0 -- which
+# reads as "host out of memory" when the host actually has 240 GiB free.
+# Everything below parses on a colon split, which is tab-agnostic, and the
+# memory figure is cross-checked before it is shown.
+set -uo pipefail
+
+MB=1048576
+GLM_CTR="${GLM_CTR:-glm-w4a16}"
+
+mem_field() { awk -v k="$1:" '$1==k {printf "%.1f", $2/'"$MB"'; exit}' /proc/meminfo; }
+section() { printf '\n== %s ==\n' "$1"; }
+
+section "time / host"
+printf '  %s   %s\n' "$(date +%H:%M:%S)" \
+  "$(awk -F: '/model name/{print $2; exit}' /proc/cpuinfo | sed 's/^ *//')"
+
+section "memory (GiB)"
+TOT=$(mem_field MemTotal); AVAIL=$(mem_field MemAvailable)
+printf '  MemTotal     %8s\n' "$TOT"
+printf '  MemAvailable %8s\n' "$AVAIL"
+printf '  Cached       %8s\n' "$(mem_field Cached)"
+awk -v t="$TOT" -v a="$AVAIL" 'BEGIN{
+  if (t+0 <= 0 || a+0 <= 0) { print "  *** PARSE BUG: zero value ***"; exit }
+  if (a+0 > t+0)         { print "  *** PARSE BUG: available > total ***"; exit }
+  printf "  check        ok (%.0f%% free)\n", 100*a/t
+}'
+
+section "swap"
+swapon --show=NAME,SIZE,USED --noheadings 2>/dev/null \
+  | awk '{printf "  %-12s %6s used of %s\n",$1,$3,$2}' || echo "  none"
+
+section "gpu"
+# CSV layout: device,VRAM Total Memory (B),VRAM Total Used Memory (B)
+rocm-smi --showmeminfo vram --csv 2>/dev/null | awk -F, '
+  NR>1 && $1 ~ /^card[0-9]+$/ {
+    printf "  %-8s %7.2f GiB used of %.2f\n", $1, $3/1073741824, $2/1073741824
+  }'
+printf '  compute util: '
+rocm-smi --showuse --csv 2>/dev/null | awk -F, 'NR>1 && $1 ~ /^card[0-9]+$/ {printf "%s=%s%% ", $1, $2}'
+echo
+
+section "disk"
+df -h /home/dank 2>/dev/null | awk 'NR==2{printf "  %s  %s used  %s free  (%s)\n",$1,$3,$4,$5}'
+
+section "llm containers"
+if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qE 'glm|frank'; then
+  podman ps -a --format '{{.Names}}|{{.Status}}' 2>/dev/null \
+    | awk -F'|' '$1 ~ /glm|frank/ {printf "  %-18s %s\n",$1,$2}'
+else
+  echo "  none"
+fi
+
+section "frank-primary unit"
+printf '  active:  %s\n' "$(systemctl is-active frank-primary 2>/dev/null | head -1)"
+printf '  enabled: %s\n' "$(systemctl is-enabled frank-primary 2>/dev/null | head -1)"
+
+section "glm serve tail"
+if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qx "$GLM_CTR"; then
+  podman logs "$GLM_CTR" 2>&1 | grep -viE 'multiproc_executor\.py:943' | tail -5 | sed 's/^/  /'
+else
+  echo "  (container $GLM_CTR not present)"
+fi
+
+section "done"
@@ -1077,19 +1077,26 @@ deploy_anubis() {
    wait_container_healthy anubis 60 || return 1
 }

-# frank-large is the stock llama.cpp prebuilt (unsloth b11030, ROCm
-# gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the DFlash2
-# drafter (z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M) on the Strix Halo iGPU,
-# as a ROOT unit since the 2026-09-19 cutover: the runtime is a prebuilt
-# binary, so there is no container image to install and no generated unit
-# to dodge. relink_unit points /etc/systemd/system/frank-large.service at
-# the repo file and boot persistence is that unit's [Install] section.
+# frank-large is gufo (gufo-org/gufo), the Strix Halo inference engine built
+# from the pinned v0.1.1 tag by scripts/build-gufo.sh, serving
+# unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the DFlash2 drafter
+# (z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M) on the Strix Halo iGPU. It is a
+# ROOT unit: the runtime is a native binary linked against the host ROCm,
+# so there is no container image to install and no generated unit to dodge.
+# relink_unit points /etc/systemd/system/frank-large.service at the repo
+# file and boot persistence is that unit's [Install] section.
 #
-# It replaces the Kairic Edge runtime (ciru-ai/ROCmFPX), which had
-# replaced the unsloth llama.cpp prebuilt serving
-# unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS. The port gate is the same as
-# every cutover since 2026-09-16: nothing starts until 7770 is released.
-LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-UD-Q8_K_XL.gguf}"
+# The runtime gate matters as much as the payload gate: gufo is built from
+# source on this host (there is no prebuilt release), so a machine whose
+# /home/dank/GitPackages tree was never built has a unit that can never
+# become healthy, and a green deploy would leave the endpoint dark.
+#
+# It replaces the stock llama.cpp prebuilt (unsloth b11030), which had
+# replaced the Kairic Edge runtime (ciru-ai/ROCmFPX). The port gate is the
+# same as every cutover since 2026-09-16: nothing starts until 7770 is
+# released.
+LLM_LARGE_RUNTIME="${LLM_LARGE_RUNTIME:-/home/dank/GitPackages/gufo-v0.1.1/bin/gufo}"
+LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf}"
 LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf}"

 retire_halogen_large_quadlet() {
@@ -1126,6 +1133,11 @@ deploy_llm_large() {
    # Refuse rather than restart-loop. A unit whose runtime or payload is
    # missing never becomes healthy, and a green deploy would leave the
    # endpoint dark with nothing in the deploy log to say why.
+    if [[ ! -x "$LLM_LARGE_RUNTIME" ]]; then
+        echo "ERROR: gufo runtime absent at $LLM_LARGE_RUNTIME" >&2
+        echo "frank-large: build it: bash scripts/build-gufo.sh" >&2
+        return 1
+    fi
    local missing=0 f
    for f in "$LLM_LARGE_MODEL" "$LLM_LARGE_DRAFT"; do
        if [[ ! -f "$f" ]]; then
@@ -1139,13 +1151,19 @@ deploy_llm_large() {
    retire_halogen_large_quadlet
    # On a re-deploy the predecessor is THIS unit: stop it before waiting on
    # the port, or the wait deadlocks on the server it just stopped serving.
-    # Then sweep a stray manual run of the SAME model -- that is how a bench
-    # harness leaves the port busy. Match the model path, not the binary name:
-    # frank-small on this host is also a llama-server and must not be killed.
-    # The pattern never appears in the deploying shell's own command line, so
-    # -f is safe here (it is not in `bash deploy-remote.sh`).
+    # Then sweep a stray manual run -- that is how a bench harness leaves the
+    # port busy. Match the DRAFTER path, which every frank-large command
+    # line carries via --dflash-model and which is UNCHANGED by the 2026-09-28
+    # payload swap: the target moved to a different repository while the
+    # drafter stayed put, so keying on the target filename would have quietly
+    # stopped matching. It is also the right key in principle — it names this
+    # endpoint's speculative drafter, not a binary. Match that, not a binary
+    # name: frank-small on this host serves a different GGUF over llama-server
+    # with no DFlash2 drafter and must not be killed. The pattern never
+    # appears in the deploying shell's own command line, so -f is safe here
+    # (it is not in `bash deploy-remote.sh`).
    run_privileged systemctl stop frank-large.service 2>/dev/null || true
-    pkill -f 'Qwen3.8-27B-UD-Q8_K_XL' 2>/dev/null || true
+    pkill -f 'Qwen3.8-27B-DFlash2-Q4_K_M' 2>/dev/null || true
    # Nothing may start until the predecessor has released the port. On
    # 2026-09-16 the cutover failed exactly here: the old server logged
    # "Deactivated successfully", the next engine tried to bind 7770 a second
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Download the Swift-1.5 frank-large candidate on Dankclaw:
+#   ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
+#     Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
+#
+#   bash scripts/download-frank-large-swift.sh
+#   bash scripts/download-frank-large-swift.sh --verify-only
+#
+# Integrity: the file is checked against its declared byte count and sha256
+# (the LFS oid from the Hub, 2026-09-28). NO AUTH HEADER IS USED: the repo is
+# public and ungated, and a token on the curl command line is world-readable
+# via /proc/<pid>/cmdline. If this ever needs a token, put it in a mode-600
+# file and pass it with `curl -K`.
+#
+# Layout after a successful run (MODEL_DIR):
+#   Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf  31457990272
+#
+# This is the CANDIDATE payload, not the deployed one. frank-large still
+# serves scripts/download-frank-large.sh's output; do not repoint
+# frank-large.env at MODEL_DIR until the gufo load test passes.
+set -euo pipefail
+
+MODEL_DIR="/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL"
+REL="Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf"
+SIZE=31457990272
+SHA=9076f99a2b1c0d232d5c169bb117e919ca21bafbe092be8fc8d9c4d645ed09c9
+URL="https://huggingface.co/ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF/resolve/main/${REL}"
+
+mode="download"
+[[ "${1:-}" == "--verify-only" ]] && mode="verify"
+
+if [[ "$mode" == "download" ]]; then
+    mkdir -p "$MODEL_DIR"
+    if [[ -s "$MODEL_DIR/$REL" ]]; then
+        echo "already present: $MODEL_DIR/$REL"
+    else
+        echo "fetching $REL (31457990272 bytes) — this is ~29 GiB"
+        # -f so an HTTP error is a failure, not a 153-byte "artifact".
+        # -C - so an interrupted run resumes instead of restarting.
+        curl -fL -C - --retry 5 --retry-delay 3 -o "$MODEL_DIR/$REL" "$URL"
+    fi
+fi
+
+echo "verifying $MODEL_DIR against declared size + sha256"
+python3 - "$MODEL_DIR" "$REL" "$SIZE" "$SHA" <<'PY'
+import hashlib, os, sys
+model_dir, rel, size, sha = sys.argv[1], sys.argv[2], int(sys.argv[3]), sys.argv[4]
+dest = os.path.join(model_dir, rel)
+if not os.path.isfile(dest):
+    print(f"  MISSING {rel}")
+    sys.exit(1)
+actual = os.path.getsize(dest)
+if actual != size:
+    print(f"  SIZE MISMATCH {rel}: {actual} != {size}")
+    sys.exit(1)
+h = hashlib.sha256()
+with open(dest, "rb") as f:
+    for chunk in iter(lambda: f.read(1 << 24), b""):
+        h.update(chunk)
+if h.hexdigest() != sha:
+    print(f"  SHA256 MISMATCH {rel}: {h.hexdigest()}")
+    sys.exit(1)
+print(f"  OK {rel} ({actual} bytes, sha256 matches the Hub LFS oid)")
+PY
+
+echo "swift candidate payload complete: $MODEL_DIR"
@@ -1,6 +1,5 @@
-# frank-large — stock llama.cpp (mainline, the unsloth b11030 prebuilt)
-# serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL on the Strix Halo iGPU
-# (gfx1151).
+# frank-large — gufo (gufo-org/gufo) serving
+# Swift-1.5-Qwen3.8-27B-Uncensored (abliterated) on the Strix Halo iGPU.
 #
 # These are the tunables read by scripts/serve-frank-large.sh. The flag
 # list is owned by the serve script; this file only sets fleet values.
@@ -9,28 +8,38 @@
 # runtime reads those bare (the 2026-08-13 TEMP incident). HOST is the one
 # exception because the serve script itself consumes it for --host.

-# The payload (scripts/download-frank-large.sh): the serve script refuses to
-# start without the main GGUF and the DFlash2 drafter.
-MODEL_DIR=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL
+# The target payload (scripts/download-frank-large-swift.sh): the serve
+# script refuses to start without the main GGUF and the DFlash2 drafter.
+# Swapped in from unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL on 2026-09-28. The two
+# are structurally interchangeable for gufo (same qwen35 architecture, the
+# same 866 tensors in the same order, the same per-tensor Q8_0/F32/BF16
+# layout, the same MTP head), so only this path changed.
+MODEL_DIR=/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL
+MODEL_FILE=/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf

-# The stock llama.cpp runtime (the unsloth b11030 prebuilt).
-RUNTIME_DIR=/home/dank/GitPackages/unsloth-llama-b11030
+# The DFlash2 drafter is UNCHANGED by the model swap and still lives beside
+# the previous target, so it carries its own directory. It has a sliding
+# window of 2048, which is what keeps its KV small (~40 MiB).
+DRAFTER_DIR=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL
+DRAFTER_FILE=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf

-# Fleet endpoint: the same port and alias the prior deployments served, so
+# The gufo runtime, built from source by scripts/build-gufo.sh.
+RUNTIME_DIR=/home/dank/GitPackages/gufo-v0.1.1
+
+# Fleet endpoint: the same port and alias every frank-large has served, so
 # no client, probe or models.yml entry changes address.
+ALIAS=frank-large
 HOST=0.0.0.0
 PORT=7770

-# One stream, full 262144 context (the model's native window).
+# One session, full 262144 context (the model's native window).
 CONTEXT_SIZE=262144
-PARALLEL_SLOTS=1
+SESSIONS=1

-# DFlash2 spec decode (a block-diffusion drafter) at depth 4. The drafter
-# has a sliding window of 2048 (small draft KV, ~40 MiB).
-SPEC_DRAFT_N_MAX=4
-
-# Prompt-cache budget: 8 GiB. The model is a 27B hybrid (16 full attention
-# + 48 Gated DeltaNet linear attention); its DeltaNet state is smaller than
-# a primarily-DeltaNet model, so the fleet's 12.8 GiB DeltaNet floor does
-# not apply.
-CACHE_RAM=8192
+# DFlash2 spec decode (a block-diffusion drafter). gufo sizes the draft
+# block itself; DRAFT_TOKENS caps the proposals per step (gufo's default is
+# 7, chosen over llama.cpp's fixed depth 4 because gufo's adaptive policy
+# shortens the block when acceptance drops) and DRAFT_POLICY is the block
+# length controller.
+DRAFT_TOKENS=7
+DRAFT_POLICY=adaptive
@@ -1,5 +1,5 @@
 [Unit]
-Description=stock llama.cpp (unsloth b11030, ROCm gfx1151) Qwen3.8-27B UD-Q8_K_XL, DFlash2 spec decode, fp8 KV, 262144 ctx / 1 slot, root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
+Description=gufo (gufo-org/gufo, ROCm gfx1151) Qwen3.8-27B UD-Q8_K_XL, DFlash2 spec decode, 262144 ctx / 1 session, root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
 After=network-online.target
 Wants=network-online.target

@@ -44,4 +44,9 @@ R9V_FULL_MUTABLE_CACHE=0
 # Braces in values get the same single-quote law as parentheses (source
 # parses assignment values).
 R9V_CHAT_TEMPLATE_FILE=/home/dank/serverAppConfigs/scripts/qwen-sharp-v22.4.0.jinja
-R9V_CHAT_TEMPLATE_KWARGS='{"preserve_thinking":true}'
+R9V_CHAT_TEMPLATE_KWARGS='{"preserve_thinking":true}'
+# Prefix caching (bench matrix E8, 2026-09-28): qualified on the q8 profile
+# at 131072, zero VRAM cost (reuses freed KV blocks), cold path unchanged.
+# Reuse-heavy traffic (harness retries/resends) hits from the third round:
+# 32k TTFT 44.2 s -> 3.6 s, 131k 439 s -> 6.9 s. Reversible: flip to 0.
+R9V_ENABLE_PREFIX_CACHING=1
@@ -1,32 +1,52 @@
 #!/usr/bin/env bash
-# Serve unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL as frank-large on
-# Dankclaw's Strix Halo iGPU (gfx1151), using stock llama.cpp
-# (mainline, the unsloth b11030 prebuilt).
+# Serve Swift-1.5-Qwen3.8-27B-Uncensored (abliterated) as frank-large on
+# Dankclaw's Strix Halo iGPU (gfx1151), using gufo (gufo-org/gufo) — the
+# inference engine built specifically for this hardware.
 #
-# Runtime: /home/dank/GitPackages/unsloth-llama-b11030/llama-server
-# (mainline, ROCm gfx1151 prebuilt). The build supports the qwen3_5
-# architecture and the DFlash2 drafter (llama_model_qwen35,
-# llama_model_dflash, common_speculative_impl_draft_dflash).
+# Runtime: /home/dank/GitPackages/gufo-v0.1.1/bin/gufo, built from source
+# against the host ROCm by scripts/build-gufo.sh. It replaced the stock
+# llama.cpp prebuilt (unsloth b11030) on 2026-09-28; port, alias and client
+# contract did not move.
 #
-# Shape: one stream, full 262144 context (the model's native window), all
-# layers on the iGPU (--n-gpu-layers 999). The 27B dense model (31.46 GiB GGUF) + the
-# DFlash2 drafter (1.14 GiB) + q8_0 KV (~8.6 GiB) + compute buffers fits the
-# iGPU's GTT aperture (~114 GiB).
+# Payload: ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF's
+# UD-Q8_K_XL, fetched by scripts/download-frank-large-swift.sh. It replaced
+# unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL on 2026-09-28. The two are structurally
+# interchangeable for this engine — same `qwen35` architecture, the same 866
+# tensors in the same order, the same per-tensor Q8_0/F32/BF16 layout, the
+# same MTP head at blk.64 — so the swap needed no engine change. The fine-tune
+# is abliterated (uncensored) and carries the SWIFT OPEN LICENSE 1.0, which
+# is not an OSI licence.
 #
-# Spec method: DFlash2 (a block-diffusion drafter) at depth 4. The drafter
-# has a sliding window of 2048 (small draft KV, ~40 MiB).
+# Shape: one session, full 262144 context (the model's native window). The
+# 27B dense weights (29.3 GiB) + the DFlash2 drafter (1.07 GiB) + KV +
+# compute buffers fit the iGPU's GTT aperture (~114 GiB); a load at this
+# context measured 66.5 GiB of the 116.7 GiB device.
 #
-# KV: q8_0 quantized (--cache-type-k q8_0 --cache-type-v q8_0). The unsloth b11030
-# build does not support fp8 KV (allowed: f32/f16/bf16/q8_0/q4_0/q4_1/iq4_nl/q5_0/
-# q5_1); q8_0 is the highest-precision 8-bit option and preserves the ~8 GiB footprint.
+# Spec method: DFlash2 (a block-diffusion drafter), gufo's adaptive block
+# policy, capped at DRAFT_TOKENS proposals per step. The drafter is the
+# z-lab Qwen3.8-27B-DFlash2-Q4_K_M and it has a sliding window of 2048
+# (small draft KV). It is UNCHANGED by the model swap and still lives beside
+# the previous target, which is why DRAFTER_DIR is its own tunable.
 #
-# Text-only (no mmproj) — matching the prior frank-large.
+# Sampling: greedy (--temperature 0) server-side, as every frank-large has
+# served. The harness overrides it per request via extraBody (1.0 / 0.95 /
+# top-k 20).
 #
-# The Qwen3.8 family template keeps prior thinking blocks across turns when
-# preserve_thinking is set; pass --jinja and the kwarg so multi-turn agents
-# do not lose the trail. The template gates reasoning_effort to {xhigh,
-# medium, low} — the harness folds its levels onto that set in models.yml
-# (reasoningEffortMap).
+# Template: gufo compiles the official Qwen Jinja into the binary and takes
+# NO --chat-template-file — and it is not a Jinja engine at all
+# (src/models/qwen/chat_template.cpp is a hand-written C++ renderer), so the
+# fleet's scripts/qwen-sharp-v22.4.0.jinja cannot be supplied to it. That law
+# was given up when this endpoint moved to gufo on 2026-09-28, by decision:
+# the terse system-prompt injection, the in-band <|think_*|> control, the
+# json|xml tool_call_format switch and the tool-argument truncation are all
+# absent. gufo's own renderer keeps the effort instruction strings and
+# preserve_thinking, which are the parts the harness depends on. Accepted
+# 2026-09-28; recovering the terse prompt from the client side is the
+# cheapest partial restore.
+#
+# Text-only: gufo auto-discovers a mmproj beside the model and this payload
+# has none (the base repo ships one separately, unused), so the endpoint
+# stays text-only.
 #
 # Tunables come from frank-large.env unless FRANK_LARGE_ENV_FILE overrides
 # it; anything not set there falls through to the defaults below.
@@ -41,67 +61,61 @@ if [[ -f "$ENV_FILE" ]]; then
 fi

 # --- defaults (used when the env file does not set them) ------------------
-: "${RUNTIME_DIR:=/home/dank/GitPackages/unsloth-llama-b11030}"
-: "${MODEL_DIR:=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL}"
+: "${RUNTIME_DIR:=/home/dank/GitPackages/gufo-v0.1.1}"
+: "${MODEL_DIR:=/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL}"
+: "${ALIAS:=frank-large}"
 : "${HOST:=0.0.0.0}"
 : "${PORT:=7770}"
 : "${CONTEXT_SIZE:=262144}"
-: "${PARALLEL_SLOTS:=1}"
-: "${SPEC_DRAFT_N_MAX:=4}"
-: "${CACHE_RAM:=8192}"
-TEMPLATE_FILE="${TEMPLATE_FILE:-$SCRIPT_DIR/qwen-sharp-v22.4.0.jinja}"
+: "${SESSIONS:=1}"
+: "${DRAFT_TOKENS:=7}"
+: "${DRAFT_POLICY:=adaptive}"

-SERVER_BIN="$RUNTIME_DIR/llama-server"
-MODEL_FILE="$MODEL_DIR/Qwen3.8-27B-UD-Q8_K_XL.gguf"
-DRAFTER_FILE="$MODEL_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf"
+GUFO_BIN="$RUNTIME_DIR/bin/gufo"

-if [[ ! -x "$SERVER_BIN" ]]; then
-    echo "runtime not found at $SERVER_BIN" >&2
+# The target GGUF and the drafter are named explicitly because they do not
+# share a directory any more: the target is a different repository's artifact
+# and keeps its own long filename, while the DFlash2 drafter is unchanged and
+# still lives beside the previous target. MODEL_FILE/DRAFTER_FILE default to
+# the deployed payload, so the env file only has to state them when it moves.
+: "${MODEL_FILE:=$MODEL_DIR/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf}"
+: "${DRAFTER_DIR:=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL}"
+: "${DRAFTER_FILE:=$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf}"
+
+if [[ ! -x "$GUFO_BIN" ]]; then
+    echo "runtime not found at $GUFO_BIN" >&2
+    echo "frank-large: build it: bash scripts/build-gufo.sh" >&2
    exit 1
 fi
 for f in "$MODEL_FILE" "$DRAFTER_FILE"; do
    if [[ ! -f "$f" ]]; then
-        echo "missing asset $f (run scripts/download-frank-large.sh)" >&2
+        echo "missing asset $f (run the matching scripts/download-frank-large*.sh)" >&2
        exit 1
    fi
 done

-# --- env (the unsloth build's ROCm setup) --------------------------------
+# --- env -------------------------------------------------------------------
+# gufo links against the host ROCm through its RPATH (/opt/rocm ->
+# /etc/alternatives/rocm), so no LD_LIBRARY_PATH is needed. The device is
+# already gfx1151 — `gufo diagnose` reports it natively — so no
+# HSA_OVERRIDE_GFX_VERSION either; the override existed for the prebuilt
+# llama.cpp runtime and can only misreport the arch here.
 export HIP_VISIBLE_DEVICES=0
-export HSA_OVERRIDE_GFX_VERSION=11.5.1
-export LD_LIBRARY_PATH="/opt/rocm/lib:/opt/rocm/llvm/lib:${LD_LIBRARY_PATH:-}"

 # --- the serve command ---------------------------------------------------
-ARGS=(
-    --host "$HOST"
-    --port "$PORT"
-    --model "$MODEL_FILE"
-    --alias frank-large
-    --ctx-size "$CONTEXT_SIZE"
-    --parallel "$PARALLEL_SLOTS"
-    --n-gpu-layers 999
-    --jinja
-    --metrics
-    --chat-template-file "$TEMPLATE_FILE"
-    --chat-template-kwargs '{"preserve_thinking":true}'
-    --cache-type-k q8_0
-    --cache-type-v q8_0
-    --cache-ram "$CACHE_RAM"
-    # DFlash2 spec decode (a block-diffusion drafter)
-    --spec-type draft-dflash
-    --model-draft "$DRAFTER_FILE"
-    --spec-draft-n-max "$SPEC_DRAFT_N_MAX"
-    --spec-draft-n-min 0
-    --spec-draft-device ROCm0
-    # Greedy sampling (the DFlash2 drafter needs greedy verification)
-    --temp 0
-    --top-p 1.0
-    --top-k 0
-    --min-p 0.0
-    # Reasoning (thoughts land in reasoning_content)
-    --reasoning auto
-    --reasoning-format deepseek
-    --reasoning-budget -1
-)
-
-exec "$SERVER_BIN" "${ARGS[@]}"
+# Server options precede the subcommand, model options follow it.
+exec "$GUFO_BIN" serve \
+    --host "$HOST" \
+    --port "$PORT" \
+    --sessions "$SESSIONS" \
+    llm \
+    --model "$MODEL_FILE" \
+    --served-model-name "$ALIAS" \
+    --context "$CONTEXT_SIZE" \
+    --speculative dflash2 \
+    --dflash-model "$DRAFTER_FILE" \
+    --draft-tokens "$DRAFT_TOKENS" \
+    --draft-policy "$DRAFT_POLICY" \
+    --temperature 0 \
+    --think auto \
+    --preserve-thinking on
@@ -47,9 +47,15 @@ fi

 # --- defaults (used when the env file does not set them) -------------------
 : "${LLAMA_BIN:=/home/dank/GitPackages/llama.cpp-mainline/build-cuda/bin/llama-server}"
-: "${MODEL_DIR:=/home/dank/models/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp}"
-: "${MODEL_FILE:=Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf}"
+: "${MODEL_DIR:=/home/dank/models/Gemma4-12B-QAT}"
+: "${MODEL_FILE:=Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf}"
 : "${MMPROJ_PATH:=$MODEL_DIR/mmproj-Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced-BF16.gguf}"
+# NOTE: the fallback above was the retired ISTA-DASLab Qwen3.8-27B GSQ-RCO
+# payload, which frank-small served on 2026-09-16/17 and which no longer
+# exists on disk. frank-small.env has always overridden it, so the drift was
+# invisible — but a missing or unreadable env file would have left this
+# script pointing at a payload that is not there, turning a config problem
+# into a "missing asset" that names the wrong model.
 : "${DRAFT_FILE:=}"
 : "${ALIAS:=frank-small}"
 : "${BIND_ADDR:=0.0.0.0}"
@@ -101,8 +101,8 @@ cable between claw and server is required for 10G.

 - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
 - The GPU budget is the GTT aperture: ~114 GiB (`gtt_total` in `/sys/class/drm/card1/device/`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
- The active service is `frank-large` on port 7770: stock llama.cpp (the unsloth b11030 prebuilt) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, q8_0 KV, DFlash2 drafter). See [[frank-large-unsloth-qwen38-27b-q8xl]].
- The active runtime (the unsloth b11030 prebuilt) **bundles its own ROCm runtime** next to the binary (`RPATH=$ORIGIN/../lib`), so it has no host SDK gate — the system ROCm 7.2.4 is not on its path. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) is no longer in the serving path.
+- The active service is `frank-large` on port 7770: **gufo** (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source at `/home/dank/GitPackages/gufo-v0.1.1/`, serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, DFlash2 drafter). See [[frank-large-gufo-qwen38-27b]].
+- The active runtime **links against the host ROCm** (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH, so it needs the host HIP stack rather than a bundled one — the opposite of the retired llama.cpp prebuilt, which carried its own. Building it needs the ROCm dev packages that are installed, and two workarounds recorded in the runbook (drop `librocwmma-dev`; pin `--gcc-install-dir` to GCC 15). The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) is not in the serving path.
 - The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
 - 1.1 TB free disk space is ample for model storage.

@@ -63,16 +63,16 @@ apt lists remain on disk but no NVIDIA PCI device exists.

 | Property | Value |
 |----------|-------|
-| System RAM | 60 GiB |
-| Swap | 63 GiB |
+| System RAM | **246 GiB** (upgraded for the Lucebox DeepSeek deployment of 2026-09-26; this page said 60 GiB until 2026-09-28 — a stale figure that survived the board swap) |
+| Swap | 63 GiB (`/swapfile`) |
 | /dev/shm | 32 GiB |

 ### Storage

 | Filesystem | Size | Used | Avail | Use% |
 |------------|------|------|-------|------|
-| / (nvme0n1p2) | 915G | ~640G | ~280G | ~70% |
-| /mnt/entropy (dm-0) | 91T | 86T | 910G | 99% |
+| / (nvme0n1p2) | 1.8T | 927G | ~822G | ~54% |
+| /mnt/entropy (dm-0) | 91T | 87T | **244G** | **100%** — full; do not plan model payload here |

 Models live under `/home/dank/models`; the serving build tree under
 `/home/dank/GitPackages/llama.cpp-dflash2` (the older `charlie-ROCmFPX` tree is
@@ -0,0 +1,192 @@
+---
+type: research
+title: Frontier MoE on Dankserver — what fits, on which runtime
+description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.
+tags:
+  - dankserver
+  - r9700
+  - gfx1201
+  - moe
+  - deepseek-v4.1
+  - deepseek-v4
+  - mimo
+  - glm
+  - affinity
+  - lucebox
+  - feasibility
+timestamp: 2026-09-28
+---
+
+# Frontier MoE on Dankserver — what fits, on which runtime
+
+> **Question this page answers.** Given Dankserver's hardware, which of
+> DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on
+> which runtime, and can Affinity / vllm-radiance / FreeToken-AMD be modified
+> to run them fast?
+>
+> **Answer in one line.** V4.1-Flash does not fit this box at any
+> configuration; the engine that does run it (Lucebox) measures
+> 0.8–9.1 tok/s on R9700 because decode is bound by PCIe expert streaming
+> rather than by the GPUs. The 284B **V4-Flash** on **Affinity** is the only
+> shape that clears 35 tok/s, at 110.
+
+Everything below was measured or read from source on 2026-09-28. Claims
+carried over from the engine authors are marked as theirs.
+
+## Hardware (measured live, 2026-09-28)
+
+| Property | Value |
+|---|---|
+| GPUs | 2x AMD Radeon AI PRO R9700 (Navi 48, **gfx1201**), 32,624 MiB each = **63.7 GiB** total |
+| Interconnect | PCIe 5.0 **x8** each, CPU root ports `00:01.1` / `00:01.3`, ~31.5 GB/s per card |
+| PCIe atomics | `CAP_EXP+28.w = 0x1440` (ReqEn+) on both endpoints — RCCL collectives are legal |
+| CPU | Ryzen 7 7700X — **8 cores / 16 threads** (small; CPU expert compute is expensive) |
+| RAM | **246 GiB** + 63 GiB swap |
+| Disk | 1.8 TB NVMe. `/mnt/entropy` is a separate 91 TB ext4 at **100%** — do not plan payload there |
+
+The knowledge base was wrong about RAM: [[Dankserver System Specs]] said
+60 GiB until 2026-09-28 and has been corrected. The Lucebox runbook's 246 GiB
+was right.
+
+## The three candidates
+
+| Model | Total / active | Routed experts | Attention & KV | Working gfx1201 runtime |
+|---|---|---|---|---|
+| **DeepSeek-V4.1-Flash** | 552B / 8B prefill, 16B decode | 384 top-6, 40 layers, `moe_intermediate_size` 2304, hidden 5120 | CED 20 enc + 20 dec; CSA2 Full/Reindex/Reuse; **FP4 main KV, 890 B/token**; Engram 196B at layers 1 & 14 | Lucebox `feat/ds41-engine` only (unmerged) |
+| **MiMo-V2.6-Flash-MOPD** | 309B / 15B | 256 top-8, no shared expert, 48 layers (39 SWA / 9 GA) | hybrid SWA (window 128) + GA; 5-layer MTP | vLLM / SGLang, NVIDIA-first. No AMD path |
+| **GLM-5.3-Flash** | 320B / 18B | MoE | KDA (linear) + DSA (sparse) hybrid; mHC | **tcclaviger vLLM fork — plausible native path (2026-09-29 source reading: `glm5next` registered with an `amd/` backend dir + DSA indexer + mHC path; r4d.so carries native mxfp4/dsfp4 MoE entries). Gate: a compressed-tensors MXFP4 payload (~165 GiB, none exists publicly) + a HIP boot probe.** llama.cpp GGUF (community R9700 guide) remains the verified-but-slower path |
+
+## Sizing — the arithmetic that decides it
+
+| Quantity | Value | Note |
+|---|---:|---|
+| DS4.1 official checkpoint | **475.27 GiB** | 48 shards, summed from the HF API |
+| &nbsp;&nbsp;of which Engram | **189.13 GiB** | exactly two 94.56 GiB shards — the layer 1 and 14 lookup tables |
+| DS4.1 routed-expert params | 543.6 B | 40 x 384 x 3 x 5120 x 2304 |
+| &nbsp;&nbsp;shipped as | 269 GiB (fp4) | DeepSeek's own `expert_dtype: fp4` |
+| Lucebox `ROCMFP23.gguf` | **377.51 GiB** | 179 GiB experts (2.5/3.5 bpw) + 189 GiB Engram + ~9.5 GiB dense |
+| V4-Flash official checkpoint | 148.67 GiB | **3.2x smaller than V4.1** |
+| Affinity container for V4-Flash | 101.1 GiB | 92.9 GiB experts @ 2.875 bpw + 6.2 dense + 2.0 embed |
+
+The Engram tables are **never mapped** by Lucebox, so they do not evict the
+expert page cache — that is the one piece of good news in the 377 GiB file.
+
+### The decode wall
+
+A V4.1 token selects `6 x 40 = 240` routed experts at ~11 MB each. At the
+residency the hot stack can hold (37 of 384 per layer, 9.6%), **~2.4 GB of
+expert weights cross PCIe per token**. PCIe 5.0 x8 does ~55 GB/s real:
+
+```
+2.4 GB / 55 GB/s = 44 ms  ->  ~23 tok/s absolute ceiling
+```
+
+That is **below the 35 tok/s target before any attention, Engram or dense work
+is counted.** Raising residency is the only lever, and VRAM is what bounds it:
+63.7 GiB / 11 MB = 5,800 of 15,360 experts = 37.8% maximum, and that assumes
+VRAM holds nothing else.
+
+The engine author measured the same wall on an R9700 with 61 GB of RAM
+(DS41.md, "Measured"): 0.8 tok/s cold / 1.8 tok/s warm at a 6 GB hot budget,
+0.9 / 1.9 at 16 GB, and 9.1 tok/s for the best tuned three-tier case. They
+name the cause: *"90 to 96% of the 240 routed experts a token selects are still
+cold, and decode moves roughly 2.4 GB of expert weights per token through the
+synchronous stream path. That, not the GPU, bounds decode today."*
+
+## Affinity — a DeepSeek-V4-Flash engine, and what a V4.1 port costs
+
+[Affinity](https://codeberg.org/StillDeadcode/affinity) is *"an inference
+engine for DeepSeek-V4-Flash on one or two RDNA4 GPUs"*, C++/HIP, no framework.
+**It builds clean on this host**: CMake 3.24+, `hipcc` 23, `HIP enabled:
+gfx1201`, zero errors, producing `affinity`, `aff-quantize`, `aff-info` and
+`aff-gpucheck`. Its README claims **~110 tok/s on two R9700s** against 8 on
+one. That is 3x Frank's 35 tok/s goal, for the 284B model.
+
+The 110 works because the 101 GiB container leaves the cold expert set in a
+**pinned host pool in RAM** — the SSD tier that Affinity calls "a cliff" is
+never reached. V4.1 would need 182 GiB of experts at the same 2.875 bpw, plus
+189 GiB of Engram, against the same 246 GiB of RAM.
+
+### What a V4.1 port would reuse
+
+The config parser is **fully data-driven** — every dimension is read from
+`config.json`, so nothing is compiled in at the config layer. And the expert
+tensor naming is *identical* between the two checkpoints:
+
+```
+layers.N.ffn.experts.E.w1.weight   (11,008 in V4  = 43 x 256)
+layers.N.ffn.experts.E.w1.weight   (15,360 in V4.1 = 40 x 384)
+```
+
+So the quantiser, the `.aff` container, the expert slab, the host pool and the
+heat/placement engine — **the part that produces the 110 tok/s** — all carry
+over. Hyper-connections are the same family (`hc_mult 4`, Sinkhorn 20).
+
+### What a V4.1 port would have to write
+
+Verified against both checkpoints' real `model.safetensors.index.json` on
+2026-09-28 — not inferred from the config prose.
+
+**Tensor presence.** `aff-quantize` requires 32 per-layer tensors. V4.1
+supplies 26 plus every global (`embed`, `head`, `norm`, `hc_head_*`) and the
+whole expert set. Six are missing:
+
+| Missing in V4.1 | Replaced in V4.1 by |
+|---|---|
+| `ffn.gate.tid2eid` | the Engram tables |
+| `attn.compressor.ape` | nothing — ratio-2 is *pooled, no APE* |
+| `attn.indexer.compressor.{wkv,wgate,ape,norm}.weight` | `attn.indexer.wk` + `attn.indexer.k_norm` |
+
+**Config keys.** Of the 35 Affinity reads, V4.1 supplies **34**. The one it
+lacks is the dangerous one:
+
+| Gap | Detail |
+|---|---|
+| **Engram — fails silently** | `num_hash_layers` (V4: 3) is absent, so `cfg_int` returns its default and `model.h:36`'s `n_hash_layer = 3` survives. It would route layers 0–2 through a table that does not exist: fluent output, wrong numbers, no error. This is the only silent failure; the other four gaps fail loudly. |
+| **Ratio vocabulary is disjoint** | V4 uses `{0, 4, 128}`; V4.1 uses `{0, 1, 2}` — **no overlap in the non-zero case**. Affinity computes `coff = (ratio == 4) ? 2 : 1`, so V4.1's 18 ratio-2 layers get the ratio-1 width; `comp_ape` is requested 38 times for a tensor that does not exist; and the `ratio == 4` indexer branch fires on **0** of the 8 layers that need one (`index_source_layer_ids` = 2, 8, 14, 20, 24, 28, 32, 36) — those 8 would run dense. |
+| **CED** | A 20-layer encoder feeding a 20-layer decoder whose global KV is projected from the final encoder hidden state, through an `aligner`. Affinity's `model.cpp` is a flat layer stack. |
+| **DSpark** | V4.1's draft carries **its own** 128-expert MoE at top-3 (`dspark_n_routed_experts`, `dspark_num_experts_per_tok`), 3 stages rather than 1. Affinity's draft shares the target's slab and routing. |
+| Vision | V4.1 is multimodal; Affinity has no vision path. Text-only is an acceptable loss. |
+
+V4.1 also adds 31 config keys Affinity never reads, including the whole
+`engram_*` block, `kv_source_layer_ids`, `index_source_layer_ids` and the
+candidate block pre-selection.
+
+**Verdict.** The reusable half is the *right* half — the expert quantiser,
+the `.aff` container, the slab, the host pool and the heat/placement engine
+are what produce the 110 tok/s. The rest is 6 tensor bindings, one new ratio
+vocabulary, the CED aligner, a new draft geometry and Engram: weeks of work
+in a third-party codebase, with no cheap way to prove correctness. And it
+meets the same capacity wall regardless, because 182 GiB of experts plus
+189 GiB of Engram against 246 GiB of RAM is exactly the problem Lucebox has.
+Not advisable.
+
+## The other runtimes
+
+| Runtime | gfx1201 | Architectures | Note |
+|---|---|---|---|
+| **Affinity** | yes, built here | **V4-Flash only** — "the geometry is compiled in; there is no architecture abstraction" | 110 tok/s for 284B. Best engine for this box by an order of magnitude, wrong model |
+| **tcclaviger vLLM fork** | **yes — proven 2026-09-29** | qwen4exp measured 37.4-40.6 tok/s decode (see [[dankserver-qwen38-fp8-expert-offload-tuning]]); **`glm5next` registered** (`Glm5NextForCausalLM`/`MTP`, backend-split `amd/`/`common/`/`nvidia/`, `sparse_indexer.py` for DSA); r4d.so has **native mxfp4 + dsfp4 MoE entries** (`has_dsfp4() == True`); W4A16 MoE class (`R4dW4a16MoEExperts`) wired into the same tiered offload; TunableOp silently skipped under offload (needs `CLAV_TUNABLEOP_WITH_OFFLOAD=1`, and then corrupts numerics — do not use) | the ad hoc qwen-fp8 container's runtime; **the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead** — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels |
+| **Lucebox** | yes, image on host | Qwen 3.8, DS4; **V4.1 on the unmerged `feat/ds41-engine` branch** | The only V4.1 path. **Refuses layer split and layer-major prefill for V4.1** |
+| **radiance-vLLM** (drwolfen) | yes — the active frank-primary | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture |
+| **FreeToken-ROCm** | partial | block-FP8 safetensors only (modelopt); known `CpuMoeExecutor` issue under ROCm | cannot load a GGUF |
+| **llama.cpp** | yes | GLM-5.3 confirmed by a community R9700 guide; DS-V4.1 CED unverified | no expert LRU; **8 CPU cores** make CPU expert placement the wrong trade |
+| **TokenSpeed** | claimed | GLM-5.x incl. a 5.3-Flash recipe | NVIDIA-first, unverified on gfx1201 |
+| **KTransformers** | no | GLM-5.3 tutorial exists | built for Intel AMX + NVIDIA |
+
+The two-R9700 three-owner split (primary + secondary + stream) is **not
+available on this host**: the retired quadlet recorded that
+`LUCE_DS4_MOE_TP_INPROC` is qualified for gfx1151+gfx1201 pairs, *not*
+gfx1201+gfx1201. hip:0 takes dense weights and the hot expert stack;
+`LUCE_EXPERT_STREAM_DEVICE=1` gives hip:1 the streamed-expert slot cache.
+
+## Measured on Dankserver
+
+<!-- MEASUREMENT -->
+
+## See Also
+
+- [[frank-primary-lucebox-ds4]] — the retired Lucebox frank-primary runbook
+- [[Dual-R9700-Cooperation]] — the PCIe/atomics facts behind the board swap
+- [[Dankserver System Specs]] — the host (RAM corrected 2026-09-28)
+- [[frank-primary-r9v-radiance-qwen38-q8]] — the ACTIVE frank-primary
@@ -0,0 +1,246 @@
+---
+type: research
+title: Qwen3.8-Flash-Next FP8 on tcclaviger — expert-offload decode tuning
+description: VERIFIED 2026-09-29 by measurement. The flag-level decode-tuning campaign on the ad hoc qwen-fp8 container (orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8, tcclaviger RDNA4 vLLM fork, 2x R9700): the >35 tok/s target is MET at 37.4-40.6 tok/s by turning prefetch OFF — telemetry measured its precision at 36%, wasting ~half of all gather traffic — while the original 26.8 tok/s incumbent (prefetch on) survived every other challenger. Residual upside is the fork's gather read path (effective ~6.5-9.5 GB/s vs 22.7 GB/s the link delivers at the same slab size).
+tags:
+  - dankserver
+  - r9700
+  - gfx1201
+  - qwen3.8
+  - fp8
+  - expert-offload
+  - speculative-decoding
+  - tuning
+timestamp: 2026-09-29
+---
+
+# Qwen3.8-Flash-Next FP8 — expert-offload decode tuning
+
+> **Question this page answers.** On the tcclaviger RDNA4 vLLM fork serving
+> `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` across both R9700s, what
+> runtime flags actually move decode at a 32k working point?
+>
+> **Answer in one line.** **37.44–40.57 tok/s — the >35 target is met — by
+> deleting `--expert-prefetch 1`.** Expert-cache telemetry measured the
+> prefetcher at **36% precision** (36,720 slabs prefetched, 13,369 served):
+> ~half of all gather traffic was useless copies, and the copies it "saved"
+> would have been installs anyway. The prefetch-on incumbent (26.8 tok/s)
+> survived every other challenger measured: gather geometry, spec depth,
+> TunableOp, scheduler batch knobs, concurrency, ngram-only drafting.
+
+This is the ad hoc `qwen-fp8` container on Dankserver (rootless podman,
+image `docker.io/tcclaviger/vllm:latest`, engine
+`v0.29.0.dev0+g2bdbbc8080`), NOT the `frank-primary` systemd service —
+the R9V production unit is untouched, so no stop-and-mask was required for
+any of this. All numbers are decode tok/s measured from the server's own
+`usage.completion_tokens / (wall - ttft)` — SSE chunk counting is invalid
+under speculative decoding because rejected drafts change the accounting.
+
+## Deployment shape (the tuned incumbent)
+
+Serve script on the host (`/home/dank/serve_qwen_fp8.sh`), env hooks in
+CAPS. **PREFETCH must be unset** (omitting the flag; the fork's default is
+off):
+
+```
+MODEL=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8   (~172.8 GiB, 131 shards)
+IMAGE=docker.io/tcclaviger/vllm:latest
+--tensor-parallel-size 2  --max-model-len 32768  --gpu-memory-utilization 0.98
+--kv-cache-memory-bytes 2147483648  --max-num-seqs 4  --enable-prefix-caching
+--speculative-config '{"method":"mtp","num_speculative_tokens":4}'
+# NO --expert-prefetch: measured 36% precision, ~50% wasted gather traffic
+```
+
+Device notes carried from the bring-up: `--device /dev/dri` must be the
+WHOLE directory (individual renderD nodes enumerate but refuse contexts —
+"No CUDA GPUs are available" while device_count() reports 2); the image
+entrypoint execs `vllm serve` itself.
+
+The memory plan this produces (from the engine's own `expert_plan.py` log):
+
+| region | size |
+|---|---:|
+| expert cache space | 15.05 GiB = 6,574 slots x 2.344 MiB |
+| &nbsp;&nbsp;VRAM-only resident layers | 3.52 GiB (layers 0-2, 1,536 experts) |
+| &nbsp;&nbsp;swap region | 11.53 GiB (5,038 slots, 111-112/layer) |
+| host-backed experts | 52.74 GiB per rank, 23,040 experts |
+| KV reserve (one max-len request +8%) | 0.36 GiB of the 2 GiB pool |
+| CUDA graphs | 1.93 GiB (21 graphs; 1.32 GiB at `--max-num-seqs 2`) |
+| runtime overhead | 7.6 GiB (non-torch + buffers + activation) |
+
+## Measured arms (2026-09-29)
+
+Decode tok/s at 166 / 1087 / 4064 prompt tokens, median of 3 runs after a
+64-token warm pass, temperature 0, max_tokens 256:
+
+| Arm | 166 | 1087 | 4064 | Acceptance | Verdict |
+|---|---:|---:|---:|---:|---|
+| **MTP4, stock 8x16, PREFETCH OFF** | **40.57** | **37.44** | **37.57** | 54.0% | **NEW INCUMBENT — target met** |
+| MTP4, stock 8x16, prefetch ON | 26.80 | 25.15 | 26.81 | 60.7% | old incumbent (+51%/+40% worse) |
+| MTP4, prefetch ON (re-check) | 25.80 | 24.38 | 26.63 | 47.9% | reproduces (noise band ±1) |
+| MTP4, prefetch ON, gather 16x32 | 24.30 | 22.77 | 24.11 | 50.3% | −9% — ruled out |
+| MTP4, prefetch ON, gather 32x64 | 26.58 | 24.94 | 26.34 | 49.6% | parity — ruled out |
+| MTP3, prefetch ON | 24.47 | 24.41 | 24.44 | 66.7% | −9%, flat in ctx — ruled out |
+| MTP6, prefetch ON | 19.07 | 17.81 | 13.54 | 17.4% | collapses; crosses UVA threshold — ruled out |
+| MTP4, prefetch ON, MNBT 8192 | 29.32 | 24.12 | 23.65 | — | neutral — ruled out |
+| MTP4, prefetch ON, PREFETCH depth 2 | 25.83 | 25.06 | 25.87 | 56.0% | neutral — ruled out |
+| MTP4, prefetch ON, TunableOp w/ offload | 23.59 | (79.82*) | (79.89*) | (96.5%*) | **rejected — corrupts** |
+| 2-way concurrency, prefetch ON (2 streams) | — | — | 15.06 + 23.11 = **25.49 summed** | — | no gain vs 26.8 single |
+| ngram-only drafter | — | — | — | — | fork bug: PLE inputs unprepared |
+| 256k ctx (MAXLEN=262144, KV 4.5 / 4.03 GiB, prefetch off) | 36.65 / 36.48 | 34.21 / 34.13 | 34.77 / **35.02** | 53.0 / 39.4% | boots and serves; band 34.1-36.5 straddles 35 within noise; arena identical (5,648) at both KV sizes |
+
+\* The TunableOp numbers are invalid by construction: the tuned GEMMs
+changed generated behavior (repetition-loop, the server's
+"repetition detected" halt firing at ~140-146 tokens instead of the full
+256, acceptance jumping 60.7% → 96.5%), the 79.8 tok/s figures were
+measured on those halted degenerate completions, and the text-only check on
+a fresh prompt came back 20.01 tok/s — *below* baseline. GEMM autotuning
+that changes output text is broken tuning.
+
+The decode curve without spec decode is dead flat 12.11 tok/s from 304 to
+16,095 tokens — decode is context-independent on this shape, bound per-step
+by the expert gather, not by KV depth.
+
+## Telemetry — the measurement that found the win
+
+`R4D_LRU_TELEMETRY=1` arms per-layer counters; a torch-profiler window
+(`/start_profile` → traffic → `/stop_profile`) makes the fork log and write
+`expert_cache_rank<N>.json` beside the traces. **`torch_profiler_with_stack=false`
+is mandatory in the profiler config** — the default (true) grew a 6.5-min
+window's trace buffer ~40 GB and the kernel OOM-killed the engine core
+(anon-rss 95.2 GB, `oom-kill` in `journalctl -k`). The short window at
+166 ctx (~15 s of traffic) completed cleanly with `with_stack=false`.
+
+The fork's own numbers (102 steps, both ranks identical):
+
+```
+hit rate 0.8245 (137,084/166,262)   host installs 15,814
+read-through 13,364                 prefetched 36,720
+served by prefetch (migrations) 13,369
+```
+
+- **The LRU is good**: 82.45% hit rate overall; layers 0-2 pinned at
+  100%; swap-region layers 78-81%. The "arena holds only 27% of experts
+  → 73% miss" arithmetic was wrong — temporal reuse carries it.
+- **The prefetcher is the poison**: 36,720 slabs moved to serve 13,369
+  (36% precision). The 23,351 unused copies are ~0.7 GB/step/rank of pure
+  waste — roughly half of all gather traffic — and the 13,369 "savings"
+  would have been installs anyway. Depth-1 prefetch also cannot cover the
+  ~2,400-slab per-step working set with its 16 slots/layer (768 total).
+- Per-layer-step at a swap layer: ~30.6 routed, 24.8 hits, 3.3 installs,
+  2.6 read-through, 8.0 prefetched of which 1.7-3.3 useful.
+
+## The wall, measured
+
+- **PCIe link (pcie-link-probe):** GPUs `03:00.0`/`07:00.0` LnkCap/LnkSta
+  32GT/s x16; root ports `00:01.1`/`00:01.3` cap x8. Effective **Gen5 x8 =
+  31.5 GB/s/direction**. `rocm-smi pcie_link_speed: 25` is the ASPM idle
+  downshift, not the loaded rate.
+- **H2D (HIP probe, pinned):** 2.345 MiB (the engine's exact expert slab)
+  **22.74 GB/s**; 16 MiB 28.19; 1 GiB 28.74 (91% of ceiling). Pinned ≈
+  pageable — staging is not the lever. Single-transfer latency at slab size
+  105.7 µs vs 78 µs ideal → ~21% per-transfer overhead.
+- **Engine's effective gather rate:** ~6.5-9.5 GB/s = **30-42% of what the
+  link delivers at the same size**. The bottleneck is the fork's read path
+  (chunk/lane geometry, per-transfer cost), not the wire.
+- **Duty cycle:** peak GPU util 100%, smooth ~29% with prefetch on — each
+  step is ~100 ms of compute then hundreds of ms waiting on expert
+  streaming. rocm-smi reports *utilization*, which reads ~0 while weights
+  load during an arm's reload; VRAM is the honest panel there.
+
+## What the remaining flags cannot do (all measured)
+
+- **Gather geometry** — 16x32 and 32x64 measured no better despite the
+  fork's "tuned at the Gen4 ceiling" comment; the constants are already at
+  parity on Gen5 x8. The read-path gap is structural, not a tuning knob.
+- **Spec depth** — 4 is the optimum both ways (with prefetch on). Depth 6
+  additionally proves the UVA read-through path (`step tokens >
+  R4D_LRU_UVA_TOKENS=5`) is *slower* than copy-in.
+- **Prefetch anything** — depth 2 neutral, depth 1 the new negative; the
+  tier is 3x too small for the working set and the predictor is 36%
+  precise. `expert_prefetch_k/budget/slots` exist (defaults 20/14/16) but
+  the precision result makes tuning them pointless.
+- **Arena growth via shrink** — `UTIL=0.99` fails boot (`Free memory on
+  device cuda:1 (31.29/31.86)`); `--max-num-seqs 2` fails CUDA-graph
+  capture (`Constraints violated (L['query_start_loc'].size()[0])`); and
+  the MNBT 8192 arm already proved a ~1 GiB arena shift is decode-neutral
+  in both directions.
+- **Concurrency** — under TP=2 both streams' tokens are computed on both
+  cards, so each rank gathers the union of both streams' misses; the links
+  do not add capacity and the shared arena thrashes. Summed 2-way (25.49)
+  measured *below* single-stream (26.8) with prefetch on.
+- **ngram-only drafting** — fails at engine init:
+  `RuntimeError: PLE inputs were not prepared`; the fork prepares this
+  model's PLE inputs only on the MTP spec path. Needs a fork patch, not a
+  config.
+- **V-only q8 KV** — stock `--kv-cache-dtype` offers fp8 variants only
+  (K+V together); a V-only q8 needs a fork patch.
+
+## Fork bugs / traps found (for the engine authors or a future patch)
+
+1. `CLAV_TUNABLEOP_SWEEP=1` is silently skipped whenever expert offload is
+   active unless `CLAV_TUNABLEOP_WITH_OFFLOAD=1` is also set
+   (`tunableop_warmup.py`).
+2. TunableOp-with-offload, once forced on: +6.7 min warmup, no persistent
+   cache (container-local), and corrupt numerics (repetition loops,
+   early-EOS halts, invalid measurements). Rejected outright.
+3. The torch-profiler stop path hangs the workers on ROCm when
+   `torch_profiler_with_stack` is true (default): >10 min hangs on two
+   attempts while the AsyncLLM wrapper reports "stopped successfully" in
+   1 s; the default stack-tracing grew a 6.5-min window's buffer ~40 GB
+   and OOM-killed the engine core. With `with_stack=false` the stop
+   completes and the telemetry report runs. The telemetry counters remain
+   reachable ONLY through that profiler stop — no direct RPC trigger
+   exists (`telemetry_report` has exactly one caller, `gpu_worker.py`).
+4. ngram spec method does not prepare PLE inputs (above).
+
+## Where the headroom lives now
+
+The 35 tok/s target is met (37.4-40.6) with prefetch off. What is left:
+
+1. **Fork patch to the gather read path** — the ~6.5-9.5 → 22.7 GB/s
+   software gap at slab size. If the gather reached wire rate, decode
+   lands in the 60+ tok/s band before acceptance effects. Code work in
+   `expert_lru.py`'s gather (`R4D_LRU_GATHER_*` is read at import; the
+   copy-in path itself is compiled — see `/app/r4dhip/r4d.so`).
+2. **256k ctx — measured, final**: boots at `MAXLEN=262144` with
+   `KV_BYTES=4831838208` or `4328521728` alike — a 262,144-token window
+   needs **4.0 GiB** KV (hybrid attention + mamba pages ≈ 16 KiB/token,
+   not the 32k plan's 9,190 B/token; 3 GiB boot-refuses and the engine
+   estimates max len 184,224 at that pool; the exact-4.0 override loses
+   ~20 MB to accounting and boots as 3.98 < 4.0). Decode band
+   **34.1-36.5** across both KV sizes — the arena is **identical (5,648
+   slots)** at 4.5 and 4.03 GiB (the planner buckets; the trim went to
+   margin, not slots), so the window costs ~10% decode vs the 32k config
+   and no KV trim recovers it. The alternative that keeps the full
+   37.4-40.6 band is `MAXLEN=184224` at 3 GiB KV. Correction: the planner
+   sizes the arena AFTER the KV override (the earlier "before" reading was
+   wrong) — the arena is what remains.
+3. **2+ slots at long ctx** — the KV pool post-spec holds ~66k tokens at
+   2 GiB (~2x 32k streams); concurrency summed *below* parity at 4064 ctx,
+   so slots are a capacity feature, not a throughput one. Two full 256k
+   streams need ~8-10 GiB of KV — that fits the arithmetic only by cutting
+   the arena below usefulness; slots-at-256k is a fork-patch or
+   smaller-payload question.
+
+## Reproducing
+
+Host-side scratch scripts (2026-09-29): `/home/dank/serve_qwen_fp8.sh` (the
+serve script with all hooks), `/home/dank/mtp_measure.py` (usage-counter
+decode measure, 3 depths), `/home/dank/mtp_measure_short.py` (single depth,
+for profile windows), `/home/dank/bench2way.py` (concurrency), plus the arm
+scripts `arm_*.sh` / `sweep_*.sh` and the detached runner `/home/dank/job.sh`
+(outputs under `/home/dank/jobs/NAME.out|.rc|.done`). The measure scripts
+are `podman cp`'d into the container after every recreate (the serve script
+re-copies them; a recreate wipes container-local files).
+
+## See Also
+
+- [[dankserver-frontier-moe-feasibility]] — the sibling feasibility page
+  (which frontier MoE fits this box; the same PCIe-expert-stream wall at
+  V4.1 scale)
+- [[frank-primary-r9v-radiance-qwen38-q8]] — the ACTIVE frank-primary
+  (production is the R9V fork; this page's container is the ad hoc detour)
+- [[x4-pair-experiment]] — the same residency law measured on a narrower
+  link: streaming over PCIe loses, computing in RAM does not care
+- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
@@ -0,0 +1,339 @@
+---
+type: reference
+title: "gufo, Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)"
+description: "The active frank-large runbook — gufo (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source on Dankclaw and serving the abliterated ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL with the z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU: the from-source build and its two host workarounds, the GGUF header evidence that made the payload swap safe, the one-session / full-window shape, why the fleet's Qwen-Sharp chat template CANNOT be used on this runtime and what that costs, the client-contract delta, the measured prefill/decode trade, and the failure modes worth knowing."
+tags:
+  - llm
+  - gufo
+  - swift-1.5
+  - dankclaw
+  - frank-large
+  - strix-halo
+  - gfx1151
+  - q8
+  - dflash2
+  - rocm
+timestamp: 2026-09-28
+---
+
+# frank-large — gufo, Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (DFlash2)
+
+> **ACTIVE since 2026-09-28.** Replaces
+> [[frank-large-unsloth-qwen38-27b-q8xl]] (stock llama.cpp, the unsloth
+> b11030 prebuilt), which had replaced
+> [[frank-large-kairic-edge-qwen38-27b]]. Same host, same port (7770), same
+> alias (`frank-large`).
+>
+> **The payload also changed on 2026-09-28**, later the same day: the target
+> went from `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL` to
+> `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`'s
+> `UD-Q8_K_XL` — an abliterated (uncensored) fine-tune of the same base. See
+> [Payload](#payload) and, for what that costs, [the template
+> law](#the-template-law-qwen-sharp-is-gone).
+
+## Identity
+
+- **Endpoint:** `frank-large` at `192.168.0.143:7770` (Dankclaw).
+- **Runtime:** [gufo](https://github.com/gufo-org/gufo) — "the Strix Halo
+  inference engine", a vertical engine built for exactly this hardware
+  (Ryzen AI MAX+ 395 / Radeon 8060S, `gfx1151`, unified memory). Pinned at
+  **v0.1.1** (commit `b0f8673`), built from source on the host to
+  `/home/dank/GitPackages/gufo-v0.1.1/`.
+- **Service shape:** root systemd unit `scripts/frank-large.service`
+  (`User=dank`, `LimitMEMLOCK=infinity`), launched by
+  `scripts/serve-frank-large.sh` with tunables from
+  `scripts/frank-large.env`.
+
+gufo ships **no prebuilt Linux binaries** (the v0.1.0/v0.1.1 release assets
+are empty) and no Nix on this host, so the runtime is a source build. It is
+also young: v0.1.0 landed 2026-09-28, the same day as this cutover.
+
+## Why this engine
+
+gufo's own measurement of the two engines on the same Qwen3.8-27B targets
+(Strix Halo, `628ed18e` vs llama.cpp `b11069`, greedy, DFlash2):
+
+| Q8_K_XL, DFlash2 | gufo | llama.cpp | delta |
+|---|---:|---:|---:|
+| prefill, depth 0 | 457.05 t/s | 333.85 t/s | **+36.9 %** |
+| prefill, depth 131,072 | 237.24 t/s | 131.80 t/s | **+80.0 %** |
+| decode, depth 0 (mixed) | 16.08 t/s | 15.00 t/s | +7.2 % |
+| decode, depth 32,768 (mixed) | 14.41 t/s | 14.44 t/s | −0.2 % |
+| decode, depth 65,536 (mixed) | 10.87 t/s | 12.62 t/s | **−13.9 %** |
+| decode, depth 131,072 (mixed) | 8.87 t/s | 11.62 t/s | **−23.7 %** |
+| cold load to HTTP-ready | 8.17 s | 36.47 s | **4.5× faster** |
+| peak memory @ 262,144 ctx | 50.58 GiB | 48.60 GiB | +4 % |
+
+Read that honestly: **prefill, load time and short-context decode improve;
+deep-context decode on the 8-bit target gets worse.** The crossover on Q8 is
+around 32k of cached depth. frank-large runs one 262,144-token conversation,
+so a long agent session spends its life on the losing side of that line
+while a short turn wins. The Q4_K_XL target does not show the regression
+(gufo is +11.7 % to +17.0 % at every depth) — that is the honest reason to
+revisit the quantization separately from the runtime.
+
+## Build
+
+`scripts/build-gufo.sh` reproduces the runtime end to end. It clones the
+pinned tag, configures the `release` preset (HIP, `--offload-arch=gfx1151`),
+builds, installs, and finishes with `gufo diagnose`. Two host facts it
+encodes, both found the hard way on 2026-09-28:
+
+- **Ubuntu's `librocwmma-dev` must not be installed.** It ships a truncated
+  rocWMMA header set (the public headers without `rocwmma/internal/`) into
+  `/usr/include`, where it shadows AMD's complete copy under
+  `/opt/rocm/include`. The first build died on `'internal/accessors.hpp'
+  file not found` from `/usr/include/rocwmma/rocwmma.hpp`, and **a `-I`
+  cannot fix it**: in HIP mode the host compilation pass searches the system
+  directories first. The package has no reverse dependencies; the script
+  removes it.
+- **`--gcc-install-dir` must point at GCC 15.** Ubuntu 26.04 has
+  `/usr/include/c++/16`, and the ROCm clang picks the newest tree, whose
+  `<atomic_wait.h>` references `std::__detail::__notify_impl` at a symbol
+  version the installed `libstdc++.so.6` (GLIBCXX_3.4.35) does not export.
+  The link then fails with `undefined symbol`. Pinning the headers to the
+  GCC the runtime library came from fixes it.
+
+Verify an installed runtime with `bash scripts/build-gufo.sh --verify`, or
+`gufo diagnose` directly — it reports `[PASS]` plus the detected
+architecture, driver and queue budget.
+
+## Shape
+
+### Payload
+
+- **Model:** `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`,
+  file `UD-Q8_K_XL` — 27B dense, 31,457,990,272 bytes, fetched and
+  sha256-verified (`9076f99a…`, the Hub LFS oid) by
+  `scripts/download-frank-large-swift.sh`. It is an **abliterated
+  (uncensored)** fine-tune of Qwen3.8-27B. The repository also publishes an
+  `mmproj-BF16.gguf`, which is **not** used: this endpoint is text-only.
+- **Licence:** the repo declares `other` — the **SWIFT OPEN LICENSE 1.0**,
+  which is *not* an OSI licence. That is a real constraint on redistribution,
+  noted here so nobody discovers it later. It is fine for a private fleet.
+- **Replaced:** `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL`, still on disk and
+  still the rollback target for the payload.
+
+The two targets are **structurally interchangeable for this engine**, which
+is why the swap needed no engine change. Header-diffed directly:
+
+| | unsloth | Swift-1.5 |
+|---|---|---|
+| `general.architecture` | `qwen35` | `qwen35` |
+| tensors (count, names, order) | 866 | 866, **identical** |
+| per-tensor `ggml_type` | Q8_0×453, F32×360, BF16×53 | **identical** |
+| per-tensor dims | — | **0 mismatches** |
+| MTP head | 4 tensors at `blk.64.nextn.*` | identical |
+| geometry | 65 blk, 5120 embd, 17408 FFN, 24/4 heads | identical |
+
+The two metadata differences that look alarming are both no-ops, and both
+were confirmed against gufo's source rather than assumed:
+
+- **`general.file_type` is 15 vs 7.** gufo's `gguf_reader.cpp` states it is
+  *"deliberately not consulted… The Unsloth UD-\*_K_XL artifacts this repo
+  targets carry a stale one — UD-Q8_K_XL records ftype 15 (Q4_K_M) … The
+  tensor table always is."* The file gufo already served lies about itself
+  here, so gufo reads the tensor table.
+- **Swift adds `qwen35.attention.recurrent_layers`.** gufo never reads that
+  key anywhere in `src/`; layer kinds come from `full_attention_interval`,
+  which both files carry as `4` and which agrees with it.
+
+### Serving shape
+
+- **Context:** one session, full 262,144 tokens (the model's native window),
+  `SESSIONS=1`. The fleet's single stream is unchanged.
+- **Spec method:** DFlash2, gufo's **adaptive** block policy
+  (`--draft-policy adaptive`, `--draft-tokens 7`). The drafter is the
+  **unchanged** z-lab `Qwen3.8-27B-DFlash2-Q4_K_M` (1.07 GiB) — it did not
+  move with the target, which is why `DRAFTER_DIR` is a separate tunable
+  from `MODEL_DIR` and why the deployer's stray-run sweep keys on the
+  drafter's filename rather than the target's.
+- **Sampling:** greedy (`--temperature 0`) server-side, as every prior
+  frank-large served. The harness overrides per request via `extraBody`
+  (1.0 / 0.95 / top-k 20).
+- **Memory:** gufo's own table puts the 8-bit target at **50.58 GiB** peak
+  with context capacity 262,144, against the iGPU's ~114 GiB GTT aperture.
+
+### Measured at the 2026-09-28 cutover
+
+Live on Dankclaw. The **target** numbers are the Swift-1.5 load test
+(a standalone run under the production flags, service stopped first); the
+**engine** numbers are from the gufo cutover earlier the same day and are
+unchanged by the payload swap.
+
+- **9.2 s** `load_started` → `load_completed` for the Swift target, against a
+  20-minute `LLM_WAIT` budget. The unsloth target loaded in 13 s.
+- **66.5 GiB** of the 116.7 GiB iGPU device at context capacity 262,144;
+  host RSS 30.6 GiB. No loader warnings, no skipped sections.
+- Contract battery, all green on the Swift target: `/v1/models` →
+  `context_length 262144`, a chat completion carrying both `content` and
+  `reasoning_content`, a `tool_calls` response with
+  `finish_reason: tool_calls`, SSE streaming, and a **two-turn tool loop**
+  (tool call → tool result → follow-up) with prior reasoning preserved.
+  Arithmetic spot-check: "17*23" → `391`.
+- Behavioural spot-check that this is the abliterated model: asked for a
+  two-sentence lockpicking scene, it wrote one without hedging or refusing.
+- From the gufo cutover, on the unsloth target: a short-prompt live request
+  returned
+  `Server-Timing: ttft;dur=549.8, inter_token;dur=24.2` (≈41 tok/s).
+  Short-prompt decode is gufo's *winning* regime; see the depth table above
+  before quoting it as the endpoint's number.
+
+### Tuning: the served flags are already optimal (2026-09-28)
+
+Frank asked for iterative decode/prefill improvement at a 32k working point
+without touching the model or the quantization. **Result: no win exists at
+this engine's served settings, and nothing was changed.** Measured on real
+repo prose and shell sources at **32,762 prompt tokens**, salted per run so
+a warm prefix cache could not help, 3–5 repetitions each.
+
+| lever | range swept | outcome |
+|---|---|---|
+| `--prefill-chunk` | 512 → 8192 | **no effect** — 410.7–412.4 t/s prefill across a 16× change |
+| `--draft-tokens` | 5 → 10 | **flat** — 13.79–15.86 t/s decode, inside run-to-run noise |
+| `--draft-policy` | fixed vs adaptive | 35.90 vs 35.50 (bench) — no effect |
+| `--min-draft-tokens` | — | **rejected by the engine**: "DFlash2 requires … 1" |
+| MTP backend | — | **unsupported for Q8**: `token_embd.weight` is Q8_0 |
+| `--context` 262144 → 40960 | diagnostic | 13.93 vs 14.71 t/s — no gain, and it would break the declared client context |
+
+The confirmation round at 5 reps **reversed** the 3-rep leaders: the pair
+that looked best (`dt9 + chunk4096`) came last at 12.86 t/s, and the
+**deployed configuration came first at 14.71 t/s**. Every difference sits
+inside the noise band.
+
+What *does* move the number is speculative decoding itself: **6.65 t/s off →
+~15 t/s on, a 5.3× step** — and that is already switched on. Prefill is
+insensitive to everything tried, which is consistent with gufo's own
+`EXPERIMENTS.md`, where dozens of kernel-level alternatives are recorded as
+rejected for "no release throughput improvement".
+
+**Do not repeat this investigation without new information.** Two traps
+cost most of the time, and both are in AGENTS.md's benchmarking policy:
+`gufo bench` reports `acceptance=0.933 ± 0.000` on an identical trace hash
+and **inverts** the draft-width ranking; and a live A/B needs ≥128 generated
+tokens or `inter_token` is first-token noise.
+
+## The template law: Qwen-Sharp is gone
+
+`scripts/qwen-sharp-v22.4.0.jinja` is the fleet's chat template — it is
+wired into `frank-primary` as `R9V_CHAT_TEMPLATE_FILE` and it was
+`frank-large`'s `--chat-template-file` under llama.cpp. **frank-large can no
+longer use it**, and the reason is stronger than "gufo lacks the flag":
+
+> gufo has **no Jinja engine at all.** `src/models/qwen/chat_template.cpp`
+> is 587 lines of hand-written C++ that appends strings to build ChatML.
+> There is no `--chat-template-file`, no environment override, and no
+> template body anywhere in the binary to point at. Qwen-Sharp is 453 lines
+> of Jinja2 using `namespace()`, macros, `raise_exception` and image/video
+> branches — supplying it would mean **reimplementing it in C++**, not
+> handing gufo a file.
+
+Given the choice on 2026-09-28, the decision was to **ship on gufo's stock
+Jinja and accept the loss**. What that gives up, concretely:
+
+| Qwen-Sharp feature | On gufo |
+|---|---|
+| `terse` system-prompt injection (the headline behaviour) | **gone** |
+| in-band `<\|think_off\|>` / `<\|think_xhigh\|>` control | **gone** |
+| `tool_call_format` `json` \| `xml` switch | **gone** |
+| `max_tool_arg_chars` / `max_tool_response_chars` truncation | **gone** |
+| `add_vision_id` → `Picture N:` prefixes | **gone** |
+| effort instruction strings in the system prompt | kept (gufo has its own, same text) |
+| `preserve_thinking` | kept (`--preserve-thinking on`) |
+
+The `terse` injection is the one that matters most here, and it is worth
+being blunt about the combination: this endpoint now runs an **abliterated**
+model *without* the prompt that tells it to stop waffling. If verbosity
+shows up, the cheapest fix is to send the `terse` text as an ordinary
+**system message from the client** (`models.yml` / the harness) rather than
+through a template — that recovers the behaviour without forking the
+renderer. The full-fidelity options are reverting to llama.cpp or porting
+Qwen-Sharp into the C++ renderer; both are worse trades today, and the
+second is the same class of change that produced the 2026-09-21
+"73,600-token prompt" incident.
+
+## Client contract — what actually changed
+
+Everything the fleet depended on still holds: port 7770, alias `frank-large`,
+`reasoning_content`, `tool_calls`, streaming SSE, and text-only (gufo
+auto-discovers a `mmproj` beside the model; this payload has none).
+
+The differences worth knowing:
+
+- **No custom chat template — see
+  [The template law](#the-template-law-qwen-sharp-is-gone) above**, which is
+  the most consequential change on this endpoint. Thinking defaults to on at
+  `xhigh` with prior reasoning preserved (`--think auto
+  --preserve-thinking on`, spelled out in the serve script).
+- **`reasoning_effort` accepts the full OpenAI set** (`none`, `minimal`,
+  `low`, `medium`, `high`, `xhigh`, `max`), all verified 200 against the
+  live endpoint. Qwen-Sharp rejected anything outside `{xhigh, medium, low}`
+  with an HTTP 500, which is why `models.yml` still carries a
+  `reasoningEffortMap` fold for this model. **The fold is kept on
+  purpose**: removing it would silently move every `high` role from `xhigh`
+  to `high` effort. Drop it as its own change.
+- **Every field the harness sends is accepted.** `top_k`, `min_p`,
+  `presence_penalty`, `repetition_penalty` and `preserve_thinking` are all
+  outside gufo's documented request subset, and all returned 200. They are
+  applied from `models.yml`'s `extraBody` unchanged.
+- **No KV-cache quantization flag.** gufo picks its own KV dtype; there is
+  no `--cache-type-k`. The old `CACHE_RAM=8192` tunable is gone with it.
+  Prompt reuse is on by default, and `--cache-disk` (restart-safe) is opt-in
+  and currently off.
+- **Health/readiness split:** `GET /health` is process liveness and is what
+  the deployer gates on; `GET /ready` is model/backend readiness.
+  `GET /metrics` is Prometheus-format.
+
+- **Long bodies:** `--max-request-bytes` defaults to 8 MiB.
+
+## Failure modes worth knowing
+
+- **A missing runtime is a deploy error, not a restart loop.** The handler
+  gates on `LLM_LARGE_RUNTIME` *before* it stops anything, and the serve
+  script exits with a `build-gufo.sh` pointer rather than exec'ing.
+- **Host memory is the ceiling, and it is shared.** frank-large and ComfyUI
+  both hold gfx1151 compute queues. gufo reads
+  `/sys/class/kfd/kfd/proc/*/queues/*/type` and caps itself
+  (`GPU_MAX_HW_QUEUES`, 2 resident for `serve llm`) so a full device
+  time-slices instead of failing; four HIP processes is the device ceiling.
+- **Load is seconds, not minutes** (9.2 s measured cold on the Swift
+  target), so a redeploy no longer spends 20 minutes in `LLM_WAIT`.
+- **A payload is two files in two directories.** The target and the drafter
+  no longer live together, so "the payload is present" is not one check; the
+  deployer gates on both paths and the serve script refuses to start if
+  either is absent. A symlinked "one payload directory" would hide that.
+- **A v0.1.x runtime is a moving target.** Rebuilds are not automatic —
+  `RUNTIME_DIR` names a version, and the deployer will not silently
+  substitute another.
+
+## Rollback
+
+Two independent rollbacks, because two things changed on 2026-09-28:
+
+1. **Payload only** — repoint `MODEL_FILE` in `scripts/frank-large.env` at
+   `/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-UD-Q8_K_XL.gguf`
+   and redeploy. **The 29.3 GiB target was reclaimed on 2026-09-28**, so this
+   path now needs a re-fetch first: `scripts/download-frank-large.sh`, then
+   the env edit. The DFlash2 drafter was never touched and still lives in
+   that directory.
+2. **Engine** — [[frank-large-unsloth-qwen38-27b-q8xl]]: the unsloth b11030
+   prebuilt is still at `/home/dank/GitPackages/unsloth-llama-b11030/`.
+   Revert `RUNTIME_DIR` and the gufo-only tunables, restore the serve
+   script's llama.cpp command, and redeploy. This is also the only way to
+   get `qwen-sharp-v22.4.0.jinja` back.
+
+# Citations
+
+- Runtime: https://github.com/gufo-org/gufo (pinned `v0.1.1`, MIT)
+- Measured tables: `docs/models/qwen3.8-27b/BENCHMARKS.md` in that repo
+- API contract: `docs/SERVER.md`; flags: `docs/CLI.md`; the C++ renderer
+  that rules out a Jinja template:
+  `src/models/qwen/chat_template.cpp` (587 lines)
+- GGUF header evidence for the payload swap: both files' headers were parsed
+  and diffed directly (architecture, all 866 tensor names in order, per-tensor
+  `ggml_type` and dims) before either was loaded
+- Build: `scripts/build-gufo.sh`; serve: `scripts/serve-frank-large.sh`;
+  tunables: `scripts/frank-large.env`
+- Payload: `scripts/download-frank-large-swift.sh` (target),
+  `scripts/download-frank-large.sh` (DFlash2 drafter)
+- Predecessor: [[frank-large-unsloth-qwen38-27b-q8xl]]
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: "stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)"
-description: "The active frank-large runbook — the stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, 31.46 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) and q8_0 KV on the Dankclaw Strix Halo iGPU: the prebuilt runtime, the full-window/one-stream shape, the DFlash2 spec method, the Qwen-Sharp template, the payload, the client contract, and the failure modes worth knowing."
+description: "The RETIRED frank-large runbook (2026-09-21 → 2026-09-28; replaced by [[frank-large-gufo-qwen38-27b]], and the rollback target) — the stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, 31.46 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) and q8_0 KV on the Dankclaw Strix Halo iGPU: the prebuilt runtime, the full-window/one-stream shape, the DFlash2 spec method, the Qwen-Sharp template, the payload, the client contract, and the failure modes worth knowing."
 tags:
  - llm
  - llama.cpp
@@ -19,10 +19,12 @@ timestamp: 2026-09-21

 # frank-large — stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2)

-> **ACTIVE since 2026-09-21.** Replaces [[frank-large-kairic-edge-qwen38-27b]]
-> (Kairic Edge IU4 W4A4, now the rollback target) and
-> [[frank-large-unsloth-qwen38-flash-next]] (the Flash-Next MoE it preceded).
-> Same port (7770), same alias (`frank-large`), same client contract surface.
+> **RETIRED 2026-09-28**, replaced by [[frank-large-gufo-qwen38-27b]] (the
+> gufo engine, same weights, same port and alias); the prebuilt is intact on
+> disk as the rollback target. ACTIVE from 2026-09-21, when it replaced
+> [[frank-large-kairic-edge-qwen38-27b]] (Kairic Edge IU4 W4A4) and
+> [[frank-large-unsloth-qwen38-flash-next]] (the Flash-Next MoE it
+> preceded).

 ## Identity

@@ -40,7 +40,7 @@ The repo carries a **thin** wiring; the engine shape lives on-host:
 | Piece | File | Law |
 |---|---|---|
 | Unit | `scripts/frank-primary.service` | Root unit, `User=dank`, `LimitMEMLOCK=infinity`, `ExecStop=podman stop -t 120 frank-primary-r9v`, `Restart=on-failure` |
-| Env | `scripts/frank-primary.env` | Production overrides ONLY: bind 0.0.0.0, :7770, alias `frank-primary`, container `frank-primary-r9v`, GPU BDFs, dev-overlay adapter, `R9V_FULL_MUTABLE_CACHE=0`, sharp chat template |
+| Env | `scripts/frank-primary.env` | Production overrides ONLY: bind 0.0.0.0, :7770, alias `frank-primary`, container `frank-primary-r9v`, GPU BDFs, dev-overlay adapter, `R9V_FULL_MUTABLE_CACHE=0`, `R9V_PLE_READY_TIMEOUT=1800`, prefix caching ON, sharp chat template |
 | Serve | `scripts/serve-frank-primary.sh` | Layers setup state under repo env → execs the fork's own `scripts/launch.sh` via the docker→podman shim → `docker wait` |
 | Deployer | `scripts/deploy-remote.sh` `deploy_llm_primary` | Gates on setup state ready, image digest, PLE bytes, all six shard files + mmproj; MainPID false-green check |

@@ -140,9 +140,11 @@ edited in place — mutating it drifts the qualification.
  ssd-mmap residency).
 - Context envelope: `R9V_MAX_MODEL_LEN=131072`; harness `models.yml`
  carries `contextWindow: 131072`, `maxTokens: 36864` (compaction-gap
-  math: 70% × 131,072 = 91,750 + 36,864 = 128,614 ≤ 131,072).
- Prefix caching OFF (`R9V_ENABLE_PREFIX_CACHING=0`), CACHE192 off — the
-  KV envelope at 131,072 is smaller than the uncensored 196,608 shape.
+  math: 70% × 131,072 = 91,750 + 36,864 = 128,614 ≤ 131,072). The KV
+  envelope at 131,072 is smaller than the uncensored 196,608 shape.
+- Prefix caching OFF at cutover (`R9V_ENABLE_PREFIX_CACHING=0`), CACHE192
+  off — superseded 2026-09-28: prefix caching is ON (see bench matrix
+  below).

 ## Prefill / decode bench matrix (2026-09-28, scratch port 7773)

@@ -183,9 +185,10 @@ prompt TTFT 44.2 s → **3.6–4.3 s** (9x), 131k 439 s → **6.9 s** (63x).
 Hit pattern: rounds 1–2 miss, round 3+ hit (identical-prompt and
 conversation-growth probes both). For the harness's resend-every-turn
 traffic that is a ~40–440 s TTFT win per steady-state turn; the one-turn
-miss costs nothing extra. **RECOMMENDED for production — pending Frank's
-go-ahead** (one env line; the launcher default is already ON, the q8 state
-opted out).
+miss costs nothing extra. **DEPLOYED 2026-09-28** (Frank's go-ahead): the
+repo env sets `R9V_ENABLE_PREFIX_CACHING=1` (commit `41e9189`), verified
+on :7770 (`--enable-prefix-caching` in the container argv, health 200,
+live completion answering).

 ## Ops

@@ -94,6 +94,23 @@ frank-small, port 7771]
  (localhost/lucebox-rocm:gfx1201) serving
  Lucebox/DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB) with streamed experts —
  the fleet's only non-llama.cpp runtime detour
+- [[dankserver-frontier-moe-feasibility]] — VERIFIED 2026-09-28: which of
+  DeepSeek-V4.1-Flash (552B), MiMo-V2.6-Flash-MOPD (309B) and GLM-5.3-Flash
+  (320B) can run on the 2x R9700 box, on what runtime, and why V4.1 does not
+  fit (475 GiB payload, 189 GiB of it Engram; decode is PCIe-expert-stream
+  bound, ~23 tok/s ceiling at today's residency). Includes the Affinity
+  V4-Flash-to-V4.1 port assessment — reusable expert tiering, four
+  architecture gaps, one of them a silent-wrongness trap on the Engram key
+- [[dankserver-qwen38-fp8-expert-offload-tuning]] — VERIFIED 2026-09-29:
+  the flag-level decode-tuning campaign on the ad hoc qwen-fp8 container
+  (tcclaviger RDNA4 fork, orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8,
+  TP=2 across both R9700s). **>35 tok/s MET: 37.44-40.57 tok/s with
+  `--expert-prefetch` DELETED** — telemetry measured the prefetcher at
+  36% precision, wasting ~half the gather traffic (the LRU itself runs at
+  82.45% hit). Prefetch-on incumbent 26.8 survived every other challenger
+  (geometry, spec depth, TunableOp-corrupts, batch knobs, concurrency,
+  ngram-fork-bug); the residual gap to ~60 tok/s is the fork's gather
+  read path (6.5-9.5 GB/s effective vs 22.7 at the same slab size)
 - [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-26 (replaced by
  [[frank-primary-lucebox-ds4]], then
  [[frank-primary-llamacpp-qwen38-flash-next-q8]]): vllm-radiance 0.9.3
@@ -132,10 +149,18 @@ frank-small, port 7771]
  (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
 - [[deployment]] — deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-large-unsloth-qwen38-27b-q8xl]] — ACTIVE frank-large runbook
-  since 2026-09-21 (root systemd unit, port 7770): stock llama.cpp
-  (unsloth b11030, ROCm gfx1151) serving
+- [[frank-large-gufo-qwen38-27b]] — ACTIVE frank-large runbook since
+  2026-09-28 (root systemd unit, port 7770): gufo (gufo-org/gufo) v0.1.1, the
+  Strix Halo inference engine, built from source on Dankclaw, serving
  unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense) with the z-lab DFlash2
+  Q4_K_M drafter (adaptive block policy) on the gfx1151 iGPU — one full
+  262,144-token conversation, greedy, text-only, the two from-source build
+  workarounds, the client-contract delta, the measured prefill/decode trade
+  against llama.cpp, troubleshooting
+- [[frank-large-unsloth-qwen38-27b-q8xl]] — RETIRED 2026-09-28 (replaced by
+  [[frank-large-gufo-qwen38-27b]]; rollback target): stock llama.cpp
+  (unsloth b11030, ROCm gfx1151) serving
+  unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the z-lab DFlash2
  Q4_K_M drafter (depth 4) on the gfx1151 iGPU — one full 262,144-token
  conversation, q8_0 KV (the build lacks fp8), text-only, Qwen-Sharp
  v22.4.0 template, the 114 GiB GTT memory model, client contract,
@@ -146,9 +146,29 @@ The LLM inference fleet on the two hosts. Full index:
  computed in host RAM, the placement law, the cutover, and the rollback
 - [[frank-primary-lucebox-ds4]] — RETIRED 2026-09-26 (same-day swap;
  rollback target): the Lucebox runtime serving
-  DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB GGUF, 43-layer MoE, 256 routed
-  experts top-6) on Dankserver's R9700 with the routed experts computed
-  on CPU from the mmapped page cache — held :7770 for hours
+  DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB GGUF, **40**-layer MoE, **384**
+  routed experts top-6, 189 GiB of embedded Engram) on Dankserver's R9700
+  with the routed experts streamed from the mmapped page cache — held :7770
+  for hours
+- [[dankserver-frontier-moe-feasibility]] — VERIFIED 2026-09-28: can
+  DeepSeek-V4.1-Flash (552B), MiMo-V2.6-Flash-MOPD (309B) or GLM-5.3-Flash
+  (320B) run fast on the 2x R9700 box, and can Affinity / vllm-radiance /
+  FreeToken-AMD be modified for them? The capacity arithmetic (475 GiB
+  payload, 189 GiB of it Engram, ~2.4 GB of cold expert weights per token
+  across PCIe = a ~23 tok/s ceiling) plus the Affinity V4-Flash-to-V4.1 port
+  assessment — reusable expert tiering, four architecture gaps, one a
+  silent-wrongness trap on the Engram key
+- [[dankserver-qwen38-fp8-expert-offload-tuning]] — VERIFIED 2026-09-29:
+  the flag-level decode-tuning campaign on the ad hoc qwen-fp8 container
+  (tcclaviger RDNA4 fork, orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8,
+  TP=2 across both R9700s). **>35 tok/s MET: 37.44-40.57 tok/s with
+  `--expert-prefetch` DELETED** — telemetry measured the prefetcher at
+  36% precision, wasting ~half the gather traffic (the LRU itself runs at
+  82.45% hit); the prefetch-on incumbent 26.8 survived every other
+  challenger (geometry, spec depth, TunableOp-corrupts, batch knobs,
+  concurrency, ngram-fork-bug); the residual gap to ~60 tok/s is the
+  fork's gather read path (6.5-9.5 GB/s effective vs 22.7 at the same
+  slab size)
 - [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-26:
  vllm-radiance as ONE tensor-parallel process (TP=2 over the R4D one-shot
  allreduce) serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq (MXFP4
@@ -181,13 +201,21 @@ The LLM inference fleet on the two hosts. Full index:
  agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
  Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
  rollback record)
- [[frank-large-unsloth-qwen38-27b-q8xl]] — the ACTIVE frank-large runbook
-  since 2026-09-21 (root systemd unit, port 7770): stock llama.cpp
+- [[frank-large-gufo-qwen38-27b]] — the ACTIVE frank-large runbook since
+  2026-09-28 (root systemd unit, port 7770): gufo (gufo-org/gufo) v0.1.1,
+  the Strix Halo inference engine, built from source on Dankclaw and
+  serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (a 27B dense model) with the
+  z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU — one 262,144-token
+  conversation, adaptive block policy, greedy, text-only, the two
+  from-source build workarounds, what changed in the client contract, the
+  measured prefill/decode trade against llama.cpp
+- [[frank-large-unsloth-qwen38-27b-q8xl]] — RETIRED 2026-09-28 (replaced by
+  [[frank-large-gufo-qwen38-27b]]; rollback target): stock llama.cpp
  (unsloth b11030, ROCm gfx1151) serving
-  unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (a 27B dense model) with the z-lab
-  DFlash2 Q4_K_M drafter (depth 4) on the gfx1151 iGPU — one 262,144-token
-  conversation, q8_0 KV (the build lacks fp8), text-only, Qwen-Sharp v22.4.0
-  template, the 114 GiB GTT memory model, client contract
+  unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the z-lab DFlash2 Q4_K_M drafter
+  (depth 4) on the gfx1151 iGPU — one 262,144-token conversation, q8_0 KV
+  (the build lacks fp8), text-only, Qwen-Sharp v22.4.0 template, the
+  114 GiB GTT memory model, client contract
 - [[frank-large-kairic-edge-qwen38-27b]] — RETIRED 2026-09-21 (replaced by
  [[frank-large-unsloth-qwen38-27b-q8xl]]; same-day swap): the Kairic Edge
  runtime (ciru-ai/ROCmFPX kairic-edge-qwen38-27b-v1.2, TheRock 7.15.0,
@@ -1,8 +1,374 @@
 # Bundle Change Log

+
+## 2026-09-29
+- **frank-large tuning investigation: a measured NEGATIVE result, nothing
+  changed.** Frank asked for iterative decode/prefill improvement at a 32k
+  working point without touching the model or quantization. **No win exists
+  at gufo's served settings**, so no config change was shipped. frank-large
+  was stopped for the whole investigation (~4h) per the benchmarking policy
+  and restored healthy in 9s afterwards.
+  - Measured on the repo's own prose and shell sources at **32,762 prompt
+    tokens**, salted per run so a warm prefix cache could not flatter a
+    later run, 3–5 repetitions per configuration.
+  - **`--prefill-chunk` is dead**: 512 → 8192 moves prefill 410.7–412.4 t/s.
+    A 16x change in chunk size, 0.4% out.
+  - **`--draft-tokens` is flat**: 5 → 10 gives 13.79–15.86 t/s decode, all
+    inside the run-to-run noise band. The 5-rep confirmation round
+    **reversed** the 3-rep leaders — `dt9 + chunk4096` looked best and came
+    **last** (12.86), while the **deployed config came first** (14.71).
+  - `--draft-policy` fixed vs adaptive: no effect. `--min-draft-tokens`:
+    **rejected by the engine** ("DFlash2 requires ... 1"). MTP: **unsupported
+    for Q8 targets** (`Unsupported Qwen MTP tensor type for
+    token_embd.weight: Q8_0`).
+  - `--context 262144 → 40960` was tested as a diagnostic (the deployed value
+    preallocates KV ~8x larger than a 32k conversation needs): **no gain**
+    (13.93 vs 14.71 t/s), and it would break `models.yml`'s declared
+    `contextWindow: 262144`. Not shipped.
+  - **What actually moves the number is speculation itself: 6.65 t/s off →
+    ~15 t/s on, 5.3x** — already enabled. Prefill is insensitive to
+    everything tried, consistent with gufo's own `EXPERIMENTS.md` where
+    dozens of kernel variants are recorded rejected for "no release
+    throughput improvement".
+  - **Two traps are now in AGENTS.md's benchmarking policy**, because they
+    cost most of the wall time and both produced *false* results:
+    (1) **`gufo bench` cannot rank speculative settings and inverted the
+    ranking** — its corpus reports `acceptance=0.933 +/- 0.000` with an
+    identical trace sha256 across draft widths (the same highly-predictable
+    text every run, 93% of proposals accepted). It reported 29–37 t/s where
+    real traffic gives ~15, and made dt16 look 6% faster than dt7 when real
+    text made it 14% slower. (2) **A live A/B needs >=128 generated tokens**;
+    at 8, `inter_token` is first-token noise and the first pass of the sweep
+    produced a nonsense ordering that the 128-token pass reversed.
+  - Also worth recording: the `Server-Timing` header is
+    `ttft;dur=402.485` — the unit is in the **name** half, so splitting on
+    `=` keys the dict `ttft;dur` and every lookup silently returns nothing.
+    Four configurations reported "no Server-Timing" while the header was
+    present on all of them.
+- **dankserver FP8 decode-tuning campaign: flags exhausted at 26.8 tok/s,
+  mechanism measured.** The ad hoc qwen-fp8 container (tcclaviger RDNA4
+  fork, orcarouter Qwen3.8-Flash-Next-Uncensored-FP8, TP=2) ran a full
+  flag-level A/B battery at the 32k working point; every number is from the
+  server's usage counters (SSE chunks are invalid under spec decode). New
+  runbook: [[dankserver-qwen38-fp8-expert-offload-tuning]].
+  - Incumbent stands: MTP depth 4, stock gather 8x16, prefetch depth 1,
+    TUNABLE off — **26.80/25.15/26.81 tok/s** at 166/1087/4064 ctx,
+    acceptance 60.7%; the re-check reproduced it (25.80/24.38/26.63).
+  - **Ruled out by measurement:** gather geometry 16x32 (−9%) and 32x64
+    (parity) — the fork's "tuned at the Gen4 ceiling" comment is stale, the
+    stock constants are already right on Gen5 x8; MTP depth 3 (−9%, flat in
+    ctx) and depth 6 (collapses to 13.5–19, acceptance 17.4% — and proves
+    the UVA read-through path is SLOWER than copy-in); MNBT 8192 (neutral);
+    prefetch depth 2 (neutral — the tier is 16 slots/layer against a
+    ~2,400-slab working set at 73% misses); 2-way concurrency (summed
+    25.49 < single 26.8 — under TP both streams' misses gather on each
+    rank, the links do not add capacity, and the shared arena thrashes).
+  - **TunableOp rejected:** it silently skips under expert offload unless
+    `CLAV_TUNABLEOP_WITH_OFFLOAD=1`; once forced on it corrupts numerics —
+    repetition loops, the server's repetition-halt firing at ~140 of 256
+    tokens, "79.8 tok/s" measured on halted degenerate completions, and a
+    fresh-prompt check at 20.01 (below baseline). +6.7 min warmup, no
+    persistent cache in the container lifecycle.
+  - **Infeasible:** `UTIL=0.99` (free-memory wall: 31.29 of 31.86 GiB free
+    at worker init); `--max-num-seqs 2` (compile constraint violation at
+    CUDA-graph capture); ngram-only drafter (fork bug: "PLE inputs were not
+    prepared" — PLE prep is wired only for the MTP spec path).
+  - **The wall, measured:** the link does Gen5 x8 = 31.5 GB/s/direction;
+    the HIP probe reads 22.74 GB/s at the exact 2.345 MiB expert slab (91%
+    of ceiling at bulk); the engine's effective gather rate is ~9.5 GB/s =
+    42% of that. A decode step is ~100 ms compute + ~400 ms PCIe wait
+    (peak util 100%, smooth ~29%). The gap to the 35 tok/s target is the
+    fork's gather read path — fork-patch territory, not flags.
+  - **Telemetry fork bugs:** the torch-profiler worker stop hangs on ROCm
+    (>10 min, two attempts, while the AsyncLLM wrapper logs success in
+    1 s), and the default `torch_profiler_with_stack=true` grew a 6.5-min
+    profile window's trace buffer ~40 GB → the kernel OOM-killed the engine
+    core (anon-rss 95.2 GB, `oom-kill` in `journalctl -k`). The per-layer
+    expert-cache hit rates (`R4D_LRU_TELEMETRY=1`) are reachable ONLY
+    through that stop path, so the telemetry JSON is unreachable without a
+    fork patch.
+  - **256k ctx boots** (the last original-goal dimension): a 262,144-token
+    window needs **4.0 GiB of KV** (hybrid attention + mamba pages ≈
+    16 KiB/token, not the 32k plan's 9,190 B/token) — 3 GiB boot-refused
+    (`4.0 GiB needed > 2.98 available`; the engine's own max-len estimate
+    at that pool: 184,224). At KV 4.5 GiB it serves **36.65/34.21/34.77**
+    (arena 6,574→5,648 slots, −14%, ~10% decode cost; 2 of 3 depths just
+    under 35). KV-exact-4.0 arm in flight for the last +3.5% slots.
+    Correction recorded: the planner sizes the arena AFTER the KV override
+    (the earlier "before" reading was wrong) — the arena is what remains.
+  - **Final 256k probe: the KV trim is inert.** KV-exact 4.0 GiB
+    boot-refused by 20 MB of accounting (3.98 available < 4.0 needed;
+    engine estimate at 3.98: max len 260,176); +32 MiB (`KV_BYTES=
+    4328521728`) boots — but the arena is **identical to the 4.5 GiB arm
+    (5,648 slots)**: the planner buckets, the 0.47 GiB went to margin.
+    Decode band **36.48/34.13/35.02** (acceptance 39.4%) — same 34.1-36.5
+    band, straddling the 35 target within noise. Final 256k config:
+    `MAXLEN=262144 KV_BYTES=4328521728`, prefetch off. The full-speed
+    alternative stays `MAXLEN=184224` at 3 GiB (37.4-40.6 band).
+  - **GLM-5.3-Flash readiness REVISED by source reading (Frank's question
+    about int4 vs mxfp4):** the tcclaviger fork in the qwen-fp8 container
+    registers `Glm5NextForCausalLM`/`Glm5NextMTP` with a backend-split
+    module (`glm5next/amd|common|nvidia`, `sparse_indexer.py` for DSA);
+    `offloader/expert.py` accepts `R4dDsfp4MoEExperts, R4dW4a16MoEExperts,
+    Fp8HipMoEExperts` — and **`R4dDsfp4MoEExperts` is MXFP4 weights with
+    the DeepSeek activation law** (docstring; SwiGLU `gemm1_clamp_limit`
+    default 10.0), inheriting the mxfp4 repack/GEMM/emulation-fallback.
+    **`r4d_lib.has_dsfp4()` is True** in this build — native `moe_dsfp4` /
+    `moe_dsfp4_mt1` entries exist, so the old "MXFP4 = emulation, missing
+    kernels" GLM verdict is dead for this fork. WNA16 (int4) backend
+    oracle on ROCm reduces to HUMMING/TRITON (Marlin/TRTLLM/XPU are
+    CUDA/NVIDIA/Intel-only), payload must be compressed-tensors W4A16
+    group-wise symmetric. Remaining GLM gate: a compressed-tensors MXFP4
+    (~165 GiB) or W4A16 payload + a glm5next-on-HIP boot probe; community
+    GLM-5.3 quants are GGUF (llama.cpp path only), native FP8 is 328 GB
+    (> 246 GiB RAM). Feasibility doc rows updated.
+
 ## 2026-09-28
+- **Dankclaw model cleanup** (disk was at 93%, 137 G free). Frank asked to
+  clean up old models. **510 G staged** to
+  `~/.AGENTTRASH/20260928-retired-models/` — moved, NOT deleted, with a
+  `MANIFEST.md` giving a restore command per item. **Nothing is freed until
+  the purge is approved**; that is the point of the trash policy, and the
+  number to know before expecting the disk to move.
+  - Staged: CIRU Orca 128 G, halogen 123 G, unsloth Flash-Next IQ4_XS 91 G,
+    mradermacher IQ4_XS 91 G, unsloth 27B 29.3 G, Kairic Edge 27 G,
+    ISTA-DASLab GSQ-RCO 13 G, vLLM Gemma4 w4a16 9.6 G, vLLM Gemma4 drafter
+    838 M. Every one maps to a runtime already retired in the runbook index.
+  - **The 29.3 G rollback target went with them**, by Frank's decision. The
+    payload-only rollback is now a re-fetch first; both the runbook and
+    AGENTS.md say so. **The DFlash2 drafter was NOT reclaimed** — only the
+    `.gguf` was taken out of that directory, because the live frank-large
+    drafter lives in it. The manifest warns against restoring the directory
+    wholesale.
+  - **Kept deliberately:** `checkpoints/` 23 G. It is unreferenced by the
+    repo, but ComfyUI is live and its workflows were not searched — not
+    confirmed dead, so not taken. `logs/x4e-run.log` also stays: it is a
+    **sparse** file, 3.3 G apparent but 4 K real, so removing it frees
+    nothing.
+  - **A live reference check came first, and it earned its keep.**
+    `serve-frank-small.sh`'s *fallback* `MODEL_DIR` still pointed at the
+    retired GSQ-RCO payload frank-small served on 2026-09-16/17. The env
+    file has always overridden it, so the drift was invisible — but a
+    missing or unreadable env file would have turned a config problem into a
+    "missing asset" naming a model that no longer exists. Repointed at
+    `Gemma4-12B-QAT/`, which is what the env file already sets. Every other
+    hit was a comment, a retired fetch script's destination, or
+    `retire_halogen_large_quadlet` (which moves a *quadlet*, not a model).
+  - **Not taken, by Frank's decision — scope was models only:** podman
+    images (**279 G reclaimable, 95%**; 170 images, 23 active) and the
+    uv+pip caches (**73 G**). Together that is 352 G of regenerable garbage
+    larger than the models reclaimed, and it is the obvious next move.
+  - Pre-existing: **118 G of trash from 2026-09-21** was already staged
+    (`20260921-frank-large-assembled` 88 G, `-redownload` 30 G) and never
+    purged. Total now staged on this host: **627 G**.
+  - Verified after the moves: frank-large and frank-small both `active`,
+    both `/health` 200, and all three live assets (Swift target, DFlash2
+    drafter, Gemma4 target) still present.
+- **frank-large payload swapped to Swift-1.5 (uncensored)** (production
+  change, Dankclaw). Frank asked whether the UD Q8 of
+  `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF` would load on
+  gufo, then said to do it. Port 7770, alias `frank-large`, one 262,144
+  session, the DFlash2 drafter and the client contract are unchanged; the
+  27B dense weights are an **abliterated (uncensored)** fine-tune of the
+  same base, under the **non-OSI SWIFT OPEN LICENSE 1.0**.
+  - **Compatibility was established from the GGUF headers, not from the
+    filename.** Both files were range-fetched and parsed: same
+    `general.architecture` (`qwen35`), same 866 tensors with **identical
+    names in identical order**, **zero** per-tensor `ggml_type` or dim
+    mismatches (Q8_0×453, F32×360, BF16×53), same MTP head at `blk.64`, same
+    geometry. The two scary metadata differences are both no-ops, confirmed
+    against gufo's source: `general.file_type` (15 vs 7) is *"deliberately
+    not consulted"* by `gguf_reader.cpp` — the file gufo already served
+    records 15 (Q4_K_M) while containing no Q4_K tensor — and Swift's extra
+    `qwen35.attention.recurrent_layers` key is **never read** by gufo
+    anywhere in `src/`; layer kinds come from `full_attention_interval`,
+    which both files carry as `4` and which agrees with it.
+  - **Fetched and verified:** 31,457,990,272 bytes, sha256 `9076f99a…`
+    matching the Hub LFS oid, via the new
+    `/scripts/download-frank-large-swift.sh`. It sends **no auth header** —
+    the repo is public and ungated, and a token on the curl command line is
+    world-readable via `/proc/<pid>/cmdline`. The anonymous transfer ran at
+    ~1.8 MiB/s; resuming with the host's token through a **mode-600
+    `curl -K` config** took it to ~7 MiB/s. The download survived an ssh
+    drop by running `setsid nohup`, not under the session.
+  - **Load test, service stopped first** (the GPU policy): **9.2 s** from
+    `load_started` to `load_completed`, 66.5 GiB of the 116.7 GiB iGPU at
+    context capacity 262,144, no loader warnings. Contract battery green:
+    `context_length 262144`, `content` + `reasoning_content`,
+    `finish_reason: tool_calls`, SSE, and a **two-turn tool loop** with prior
+    reasoning preserved. Spot-checks: "17*23" → `391`; asked for a
+    lockpicking scene it wrote one without hedging, confirming the
+    abliteration.
+  - **THE TEMPLATE LAW IS GONE, by decision.** Frank asked to "use our chat
+    template"; gufo **cannot** take it. This is not a missing flag —
+    `src/models/qwen/chat_template.cpp` is 587 lines of hand-written C++
+    that appends strings, with no Jinja engine, no `--chat-template-file`,
+    no env override and no template body in the binary. Qwen-Sharp
+    v22.4.0 is 453 lines of Jinja2 with `namespace()`, macros and
+    `raise_exception`, so supplying it is a **port, not a config line**.
+    Frank chose to ship on gufo's stock renderer and accept the loss: the
+    `terse` system-prompt injection, in-band `<|think_*|>` control, the
+    `json|xml` `tool_call_format` switch, tool-argument truncation and
+    `add_vision_id` prefixes are all gone. gufo's renderer does keep the
+    effort instruction strings and `preserve_thinking`. **Worth watching:
+    an abliterated model now runs without the prompt that tells it to stop
+    waffling** — the cheap fix is to send the `terse` text as an ordinary
+    system message from `models.yml`, not to touch a template.
+  - **The payload is now two files in two directories.** The target moved
+    repo; the DFlash2 drafter did not move, so `DRAFTER_DIR` is its own
+    tunable. That broke the deployer's stray-run sweep, which keyed on the
+    *target* filename and would have silently stopped matching — a stray
+    run would then have held 7770 past the port gate. It now keys on the
+    drafter, which every frank-large command line carries via
+    `--dflash-model` and which did not move.
+  - **`/tests/frank_large_unit.bats`** — 20 assertions. The mutation pass
+    found **four real gaps** on the first run, all now fixed: two fetch
+    assertions were satisfied by a **header comment** rather than the URL
+    assignment, so repointing a repo left them green; and nothing bound the
+    env's `MODEL_FILE`/`DRAFTER_FILE` to the filenames the fetch scripts
+    actually download, so a re-upload or an env edit would break the
+    endpoint with a "missing asset" no test predicted. There is now an
+    explicit cross-file invariant. A second harness bug worth recording:
+    a `bats --filter` that matches no test reports zero `not ok`, which
+    looks exactly like a mutation that survived — the harness now asserts
+    the `1..N` count before believing a result. 16/16 mutations caught.
+  - Docs: runbook gained a **Payload** section (the header evidence),
+    **The template law** section and a two-way **Rollback**; AGENTS.md
+    fleet table, frank-large section and rollback.
+- **frank-large runtime swapped to gufo** (production change, Dankclaw).
+  Frank asked for `github.com/gufo-org/gufo` — a vertical inference engine
+  built for exactly this hardware (Ryzen AI MAX+ 395 / Radeon 8060S,
+  `gfx1151`) — in place of the stock llama.cpp prebuilt. Port 7770, alias
+  `frank-large`, one 262,144-token session and the client contract were all
+  unchanged; only the engine was new, and the payload was the unsloth
+  Qwen3.8-27B UD-Q8_K_XL **at that point** — it was swapped for the
+  abliterated Swift-1.5 later the same day, see the entry above.
+  - **The runtime is built from source on the host.** gufo ships no prebuilt
+    Linux binary (the v0.1.0/v0.1.1 release assets are empty) and the host
+    has no Nix. Pinned at `v0.1.1` / `b0f8673`, installed to
+    `/home/dank/GitPackages/gufo-v0.1.1/` by the new
+    `/scripts/build-gufo.sh`, which is reproducible and fails closed on a
+    moved tag. Unlike the unsloth prebuilt it replaces, gufo **links the host
+    ROCm 7.2.4** through its RPATH, so the HIP dev packages must stay.
+  - **Two host workarounds, both encoded in the build script.** Ubuntu's
+    `librocwmma-dev` ships a truncated rocWMMA header set into
+    `/usr/include` that shadows AMD's complete one; the build died on
+    `'internal/accessors.hpp' file not found`, and a `-I` cannot fix it
+    because in HIP mode the host compilation pass searches the system
+    directories first. The package has no reverse dependencies; it is
+    removed. Second, Ubuntu 26.04 ships `/usr/include/c++/16` and the ROCm
+    clang prefers the newest tree, whose `<atomic_wait.h>` needs a
+    `std::__detail::__notify_impl` symbol version the installed
+    `libstdc++.so.6` (GLIBCXX_3.4.35) does not export — the link failed until
+    `--gcc-install-dir` was pinned to GCC 15.
+  - `gufo diagnose` reports `[PASS]` on `gfx1151` (ROCm 7.2.3, amdgpu
+    loaded, 20 CUs). It self-caps compute queues at 2 resident for
+    `serve llm` after reading `/sys/class/kfd/kfd/proc/*/queues/*/type`,
+    which matters because ComfyUI holds queues on the same device.
+  - **Shape:** one session, `--context 262144`, DFlash2 with the **adaptive**
+    block policy capped at `--draft-tokens 7` (llama.cpp's fixed depth 4 is
+    gone), greedy, `--think auto --preserve-thinking on`, text-only.
+  - **Client-contract deltas, all in the runbook:** the chat template is
+    compiled into the binary (no `--chat-template-file`, so the Qwen-Sharp
+    73k-token-prompt failure mode cannot recur); `reasoning_effort` now
+    accepts the full OpenAI set, so `models.yml`'s `reasoningEffortMap` fold
+    is no longer required — **kept anyway** so this commit does not also move
+    every `high` role from `xhigh` to `high` effort; and there is no
+    KV-quantization or prompt-cache-RAM flag.
+  - **Measured at cutover:** 13 s from `load_started` to listening (target
+    8 s, drafter 2 s, session 3 s) against a 20-minute `LLM_WAIT` budget;
+    RSS 30.6 GiB; a live short-prompt request returned
+    `Server-Timing: ttft;dur=549.8, inter_token;dur=24.2` (~41 tok/s).
+    Verified live: `/health` 200, `/ready` 200, `/v1/models` →
+    `frank-large` at `context_length 262144`, a chat completion carrying
+    `reasoning_content`, a `tool_calls` response with
+    `finish_reason: tool_calls`, and SSE streaming.
+  - **The trade, stated plainly.** gufo's own Q8_K_XL tables against
+    llama.cpp b11069: prefill +37 % to +80 %, cold load 4.5× faster, decode
+    +7 % at zero depth — but **−14 % at 65,536 tokens of depth and −24 % at
+    131,072**. On the 8-bit target the crossover is ~32k and frank-large runs
+    one 262,144-token conversation, so long agent sessions sit on the losing
+    side of it. The Q4_K_XL target shows no such regression; revisiting the
+    quantization is a separate decision.
+  - `/tests/frank_large_unit.bats` — Rewritten around behaviour: the seam
+    runs the serve script against a stub engine and asserts on the argv it
+    actually execs, instead of grepping the script's text. A caveat found
+    while writing it: **bats 1.13 ignores a failing `A && B` in the middle of
+    a test** (bash exempts the non-final element of an AND-list from
+    `errexit`), so every such assertion was split into standalone `[ ]` lines.
+    Mutations confirmed the seam bites: dead runtime gate, `DRAFT_TOKENS=0`,
+    a wrong context/sessions/port, an alias swap, a moved drafter, a removed
+    missing-asset message and a neutered commit pin each fail it.
+  - `/scripts/deploy-remote.sh` — `deploy_llm_large` gates on the runtime
+    binary as well as the payload and returns **before** it stops the serving
+    unit, so a host whose gufo was never built reports the failure instead of
+    taking the endpoint down on the way there.
+  - Docs: new `/serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md`;
+    `frank-large-unsloth-qwen38-27b-q8xl.md` marked RETIRED (still the
+    rollback target — the prebuilt and the payload are both intact on disk);
+    AGENTS.md fleet table, frank-large section, bench policy and runbook
+    index; both vault indexes; `Dankclaw System Specs.md`; `README.md`.
+  - **e2e gate: 32/33.** The one failure is `192.168.0.112` (the FrankBook-Pro
+    Mac mini) at 100 % packet loss — offline, and the same failure recorded
+    in `564a781`. Not related to this change; no frank-large test failed.
+- **Frontier-MoE feasibility study on Dankserver** (research + build, no
+  production change). Frank asked which of DeepSeek-V4.1-Flash,
+  MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash would run best on the 2x R9700 box,
+  and whether Affinity / vllm-radiance / FreeToken-AMD could be modified to run
+  them fast. Findings, all evidenced:
+  - **V4.1 does not fit.** 475.27 GiB checkpoint (measured from the HF API),
+    of which **189.13 GiB is Engram** in two 94.56 GiB shards. A token selects
+    240 routed experts at ~11 MB; at the 37-of-384-per-layer residency the hot
+    stack can hold, ~2.4 GB crosses PCIe per token. At 55 GB/s real that is a
+    **~23 tok/s ceiling** — below the 35 tok/s goal before any attention or
+    dense work. The 1000 t/s prefill target is ~20x away: the backend refuses
+    layer-major prefill for V4.1, so the only legal prefill is token-by-token.
+  - **Affinity** (Codeberg, StillDeadcode) is a *DeepSeek-V4-Flash* engine
+    claiming 110 tok/s on two R9700s; it **builds clean here** (hipcc 23,
+    gfx1201, zero errors). The 110 works because the 101 GiB container leaves
+    the cold expert set in a pinned host pool in RAM. A V4.1 port reuses the
+    quantiser, container, slab, host pool and placement engine (expert tensor
+    naming is identical, 15,360 tensors at 40x384) but must write CED, CSA2's
+    new indexer, the DSpark draft geometry and Engram — and Engram fails
+    *silently*, because V4.1 has no `num_hash_layers` key so Affinity's
+    `model.h:36` default of 3 survives. Weeks of work, same capacity wall.
+  - **Lucebox** is the only engine that runs V4.1 on gfx1201, on the unmerged
+    `feat/ds41-engine` branch. Its own DS41.md measures 0.8-1.9 tok/s on an
+    R9700 (best tuned three-tier case 9.1) with **61 GB of RAM**; Dankserver's
+    246 GiB is the one untested lever, so the 377.5 GiB ROCMFP23 payload is
+    downloading for a three-arm bench.
+ - `/serverAppConfigs-docs/LLM/Dankserver System Specs.md` — **RAM corrected
+   60 GiB -> 246 GiB** (stale since the 2026-09-20 board swap; the Lucebox
+   runbook had it right) and storage refreshed; `/mnt/entropy` is at 100%
+   (244 G) and must not be planned for payload.
+ - `/serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md` — new
+   runbook; indexed in `LLM/index.md`.
+ - **LANDMINE, not changed here:** `/home/dank/models` was cleared by Frank
+   mid-session to free space. frank-primary is still answering `:7770` from
+   mmapped-but-unlinked inodes (`/models` is empty inside the running
+   container) — **the endpoint cannot be restarted until the 175 GiB payload
+   is re-fetched.** A reboot or a container restart takes frank-primary down.
+- Deploy-state reconciliation (Frank's call: no restart) — the autoresearch trio
+  (`f187165` PLE 1800, `41e9189` prefix caching, `ba2100e` docs) was found already
+  MERGED to main + pushed AND already live on frank-primary: the running
+  `frank-primary-r9v` container (booted 12:13 from the `ba2100e` checkout) carries
+  `--enable-prefix-caching` in argv and `VLLM_PLE_OFFLOAD_READY_TIMEOUT=1800` in env,
+  `/health` 200 — but `deploy-state/frank-primary` was still stamped `f187165`. A
+  `deploy.sh` run would have re-selected frank-primary and reloaded the 175 GiB
+  payload (~16–19 min) for zero behavioral change, on the endpoint this harness itself
+  runs on. Stamped the marker `f187165 → ba2100e` (status `deployed`, atomic
+  mktemp+mv per the deployer idiom); service untouched. e2e gate 32/33 — the lone
+  failure `11-admin` is the FrankBook-Pro Mac mini (`192.168.0.112:8989`) offline
+  (100 % ICMP loss + TCP timeout), the documented asleep-host condition, not a
+  regression.
+- `scripts/frank-primary.env`, runbook — **prefix caching DEPLOYED on
+  frank-primary** (Frank's go-ahead; commit `41e9189`): E8-qualified, zero
+  VRAM cost, 9x/63x reuse TTFT. Verified on :7770 (argv flag, health 200,
+  completion "PREFIX-CACHE-LIVE"). Gotcha re-confirmed: the container is
+  in dank's ROOTLESS podman store (`sudo podman` inspects an empty store
+  and reports nothing); probe via the shim's `docker` or the unit MainPID.
 - `/scripts/frank-primary.env`, `/serverAppConfigs-docs/LLM/frank-primary-r9v-radiance-qwen38-q8.md`, `/serverAppConfigs-docs/index.md` — **frank-primary prefill/decode bench matrix** (scratch :7773, prod stopped 00:06 for the window per fleet bench law): baseline E0 (chunk 4096 / 80-slot sync / MTP4) = PP8K 954.6, PP131K 883.0 tok/s, decode 141.9 ms/step ≈ 23.8 t/s effective. Chunk 8192 (E1R): PP8K +25.9% (1201.4) but decode −8.5% and PP131K −4.4% — short-prompt win, decode tax, not the fleet's dominant shape. Chunk 16384 boots but OOMs at request time (0.93 GiB free) — 4096 is the ceiling (engine spec-config cap + memory). Expert cache 80 slots is the policy max (doctor: capacities 48/64/80, synchronous LRU only — async unsupported, 96/112 rejected); MTP depth 6 is 3.7x SLOWER than depth 4. Prefix caching (E8/E9): zero VRAM cost, cold path unchanged, reuse hits 32k 44.2→3.6–4.3 s (9x), 131k 439→6.9 s (63x); hit pattern rounds 1–2 miss / 3+ hit — RECOMMENDED, pending Frank's go-ahead. Two failure laws recorded: unqualified shapes fail mid-request (health stays 200), and PLE registration can blow the engine's 600 s `VLLM_PLE_OFFLOAD_READY_TIMEOUT` under page-cache churn (measured 740 s) — launch.sh gained `R9V_PLE_READY_TIMEOUT` forwarding (fork `2de0357`, local-only) and prod now sets 1800.
- **Incident — frank-primary down 00:06 → ~10:10 (9h)**: the bench window's required prod stop was never restored — the matrix + follow-ups ran 00:44 → 09:02 and the session let the endpoint sit dead (the 2026-09-18 "a bench session owes the fleet a health 200" lesson, re-offended). The first restore boot (09:46) then failed on the PLE 600 s timeout above (unit "Deactivated successfully", no journal error — the serve script exits 0 on the wait); the `R9V_PLE_READY_TIMEOUT=1800` env fix + redeploy restores it. Lesson: a multi-hour bench window needs an explicit restore task, not a "watch for DONE" — DONE ≠ prod up.
+- **Incident — frank-primary down 00:06 → ~11:00 (11h)**: the bench window's required prod stop was never restored — the matrix + follow-ups ran 00:44 → 09:02 and the session let the endpoint sit dead (the 2026-09-18 "a bench session owes the fleet a health 200" lesson, re-offended). The first restore boot (09:46) then failed on the PLE 600 s timeout above (unit "Deactivated successfully", no journal error — the serve script exits 0 on the wait). Restored by the `R9V_PLE_READY_TIMEOUT=1800` fix deploy (f187165, from main — the deploy gate refuses non-main): health 200 + MainPID = the serve script's own `podman wait` + live completion verified ~11:00. Deploy e2e gate 32/33 — the one failure (11-admin) needs the FrankBook-Pro Mac mini, which was asleep and ignored two WoL bursts (240 packets); recurring pattern per the wol-wake skill. Lesson: a multi-hour bench window needs an explicit restore task, not a "watch for DONE" — DONE ≠ prod up.

 ## 2026-09-27
 - `/scripts/frank-primary.env`, `/scripts/frank-primary.service`, `/scripts/serve-frank-primary.sh`, `/scripts/deploy-remote.sh`, `/tests/frank_primary_unit.bats`, `/serverAppConfigs-docs/LLM/frank-primary-r9v-radiance-qwen38-q8.md` (new), AGENTS.md + LLM index — **frank-primary cut over to the Q8_0 payload** (uncensored IQ4_XS → unsloth Qwen3.8-Flash-Next Q8_0, 175.30 GiB / 6 shards) at Frank's direction. Launcher gained the opt-in `R9V_CHAT_TEMPLATE_FILE`/`R9V_CHAT_TEMPLATE_KWARGS` passthrough (fork commit `30d4a32`, local-only) — the Qwen-Sharp v22.4.0 template now serves, fleet parity with frank-large. Env points at `/home/dank/r9v-state/q8xl/setup.json` + the quantized-inject dev-overlay adapter (fork `363d606`) + `R9V_FULL_MUTABLE_CACHE=0`; shard gates extended to SHARD4/5/6 (unset-safe); seams recalibrated; ctx window 131072 (models.yml maxTokens 36864). Spec decode is MTP4 only — the fork's `speculative_config` is a single dict, no chaining, so llama.cpp's `draft-mtp,ngram-mod` pattern is NOT expressible (PLE n-gram rides along architecturally). Uncensored state + payload remain on disk as the rollback target. 346 tests green.
@@ -1,31 +1,95 @@
 #!/usr/bin/env bats
 # Seam: the frank-large root unit on dankclaw.
 #
-# The 2026-09-21 cutover moved this service from the Kairic Edge runtime
-# (ciru-ai/ROCmFPX) serving jcbtc/Qwen3.8-27B-IU4-Kairic-Edge onto the
-# stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) serving
-# unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the DFlash2 drafter
-# (z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M), still as a ROOT unit: a
-# prebuilt runtime has no container image to install and no generated
-# unit to dodge. This seam pins what a flag list cannot show: the
-# operator's one-stream / full-window shape, the payload the serve script
-# refuses to start without, the DFlash2 spec wiring, fp8 KV, greedy
-# sampling, a deploy handler that gates on the port, and the fact that
-# the sweep for a stray benchmark run cannot take frank-small down with
-# it.
-
+# Since 2026-09-28 this endpoint is served by gufo (gufo-org/gufo), the
+# Strix Halo inference engine built from the pinned v0.1.1 tag, serving
+# Swift-1.5-Qwen3.8-27B-Uncensored (abliterated) UD-Q8_K_XL with the z-lab
+# DFlash2 Q4_K_M drafter. Two changes landed on this endpoint that day: the
+# engine (from the stock llama.cpp prebuilt) and the payload (from
+# unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL). Port, alias and client contract did
+# not move.
+#
+# Most of this seam runs the serve script against a stub engine and asserts
+# on the argv it actually execs, so what is pinned is the command the
+# operator gets rather than the text of the script: a tunable renamed,
+# dropped, or silently defaulted away fails here the way it would fail a
+# deployment.
 load helpers

 ENV_FILE="$REPO_ROOT/scripts/frank-large.env"
 UNIT="$REPO_ROOT/scripts/frank-large.service"
 SERVE="$REPO_ROOT/scripts/serve-frank-large.sh"
-FETCH="$REPO_ROOT/scripts/download-frank-large.sh"
+FETCH="$REPO_ROOT/scripts/download-frank-large-swift.sh"
+DRAFTER_FETCH="$REPO_ROOT/scripts/download-frank-large.sh"
+BUILD="$REPO_ROOT/scripts/build-gufo.sh"
 REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
+# The deployed target's filename, kept here so a rename in the env file and a
+# rename in this stub cannot drift apart unnoticed.
+TARGET_STUB="Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf"
+setup() {
+    STAGE="$(mktemp -d)"
+    RUNTIME_DIR="$STAGE/gufo"
+    MODEL_DIR="$STAGE/models"
+    DRAFTER_DIR="$STAGE/draft"
+    mkdir -p "$RUNTIME_DIR/bin" "$MODEL_DIR" "$DRAFTER_DIR"
+    # A stub engine: the serve script execs it, so $@ is exactly what gufo
+    # would be invoked with.
+    cat > "$RUNTIME_DIR/bin/gufo" <<'STUB'
+#!/usr/bin/env bash
+printf '%s\n' "$@"
+STUB
+    chmod +x "$RUNTIME_DIR/bin/gufo"
+    # The target and the drafter live in SEPARATE directories since the
+    # 2026-09-28 payload swap: the target came from a different repository,
+    # the DFlash2 drafter did not move. A stub that colocated them would
+    # never notice DRAFTER_DIR going stale.
+    : > "$MODEL_DIR/$TARGET_STUB"
+    : > "$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf"
+    STUB_ENV="$STAGE/stub.env"
+    cat > "$STUB_ENV" <<EOF
+RUNTIME_DIR=$RUNTIME_DIR
+MODEL_DIR=$MODEL_DIR
+DRAFTER_DIR=$DRAFTER_DIR
+EOF
+}

-@test "the frank-large unit, env and serve files exist where the deployer installs them from" {
+teardown() {
+    if [ -n "${STAGE:-}" ]; then
+        rm -rf "$STAGE"
+    fi
+}
+
+# serve_argv [env-file] — run the serve script and capture the engine's argv,
+# one token per line, in $output. No argument exercises the built-in defaults
+# with the stub's paths.
+serve_argv() {
+    local envfile="${1:-$STUB_ENV}"
+    if [ -n "$envfile" ]; then
+        FRANK_LARGE_ENV_FILE="$envfile" run bash "$SERVE"
+    else
+        run env FRANK_LARGE_ENV_FILE="$STAGE/absent.env" bash "$SERVE"
+    fi
+}
+
+# argv_has <token> — exact token match, so --port never satisfies a test
+# looking for a different flag.
+argv_has() {
+    grep -qxF -- "$1" <<<"$output"
+}
+
+# argv_value <flag> — the token following <flag>, empty if the flag is absent.
+argv_value() {
+    local i
+    i="$(grep -nx -- "$1" <<<"$output" | cut -d: -f1)"
+    [ -n "$i" ] || return 0
+    sed -n "$((i + 1))p" <<<"$output"
+}
+
+@test "the frank-large unit, env, serve and build files exist where the deployer installs them from" {
    [ -f "$ENV_FILE" ]
    [ -f "$UNIT" ]
    [ -x "$SERVE" ]
+    [ -x "$BUILD" ]
 }

@test "frank-large is boot-persistent, self-healing and unprivileged" {
@@ -33,111 +97,238 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
    grep -q '^Restart=on-failure$' "$UNIT"
    grep -q '^RestartSec=' "$UNIT"
    grep -q '^User=dank$' "$UNIT"
-    # The engine pins staging buffers; the memlock budget must be unbounded
-    # or the load fails on the RLIMIT.
+    # The engine pins buffers; the memlock budget must be unbounded or the
+    # load fails on the RLIMIT.
    grep -q '^LimitMEMLOCK=infinity$' "$UNIT"
    # The unit runs the repo's own serve script with the repo's env.
    grep -q 'ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh' "$UNIT"
    grep -q 'FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env' "$UNIT"
 }

-@test "the operator's shape is one stream of the full 262144 window with DFlash2 depth 4" {
-    local ctx slots
-    ctx="$(sed -nE 's/^CONTEXT_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"
-    slots="$(sed -nE 's/^PARALLEL_SLOTS=([0-9]+)$/\1/p' "$ENV_FILE")"
+@test "the fleet endpoint and alias survive the cutover" {
+    serve_argv
+    [ "$status" -eq 0 ]
+    # frank-large at :7770, whatever engine answers it.
+    [ "$(argv_value --served-model-name)" = "frank-large" ]
+    [ "$(argv_value --port)" = "7770" ]
+    [ "$(argv_value --host)" = "0.0.0.0" ]
+}
+
+@test "the built-in defaults are one session of the full 262144 window with DFlash2" {
+    serve_argv
+    [ "$status" -eq 0 ]
    # 262144 is the model's native window (the operator's call, 2026-09-21).
-    [ "$ctx" -eq 262144 ]
-    # One stream.
-    [ "$slots" -eq 1 ]
-    # DFlash2 depth 4 (the drafter's block-diffusion depth).
-    grep -q '^SPEC_DRAFT_N_MAX=4$' "$ENV_FILE"
+    [ "$(argv_value --context)" -eq 262144 ]
+    # One session is the single stream every prior frank-large served.
+    [ "$(argv_value --sessions)" -eq 1 ]
+    [ "$(argv_value --speculative)" = "dflash2" ]
+    # A draft ceiling of 0 would silently serve the autoregressive rate.
+    [ "$(argv_value --draft-tokens)" -gt 0 ]
+}
+
+@test "the shipped env file, not just the defaults, serves the deployed shape" {
+    # The defaults are a floor; systemd sources the env file. Only the
+    # on-host paths are redirected at the stub, so every fleet value in the
+    # shipped file is the one under test. MODEL_FILE and DRAFTER_FILE are
+    # absolute in the shipped file, so they need redirecting too — a sed that
+    # only rewrote MODEL_DIR would leave them pointing at /home/dank and the
+    # test would pass for the wrong reason (or fail on a box that has them).
+    sed -e "s#^RUNTIME_DIR=.*#RUNTIME_DIR=$RUNTIME_DIR#" \
+        -e "s#^MODEL_DIR=.*#MODEL_DIR=$MODEL_DIR#" \
+        -e "s#^MODEL_FILE=.*#MODEL_FILE=$MODEL_DIR/$TARGET_STUB#" \
+        -e "s#^DRAFTER_DIR=.*#DRAFTER_DIR=$DRAFTER_DIR#" \
+        -e "s#^DRAFTER_FILE=.*#DRAFTER_FILE=$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf#" \
+        "$ENV_FILE" > "$STAGE/shipped.env"
+    serve_argv "$STAGE/shipped.env"
+    [ "$status" -eq 0 ]
+    [ "$(argv_value --served-model-name)" = "frank-large" ]
+    [ "$(argv_value --host)" = "0.0.0.0" ]
+    [ "$(argv_value --port)" = "7770" ]
+    [ "$(argv_value --context)" -eq 262144 ]
+    [ "$(argv_value --sessions)" -eq 1 ]
+    [ "$(argv_value --speculative)" = "dflash2" ]
+    [ "$(argv_value --draft-tokens)" -gt 0 ]
+    [ "$(argv_value --draft-policy)" = "adaptive" ]
+    # The shipped file must name the target the fleet actually serves, not a
+    # path that merely happens to exist.
+    [ "$(argv_value --model)" = "$MODEL_DIR/$TARGET_STUB" ]
+    [ "$(argv_value --dflash-model)" = "$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf" ]
+}
+
+@test "the engine is pointed at the payload that is actually on disk" {
+    serve_argv
+    [ "$status" -eq 0 ]
+    [ -f "$(argv_value --model)" ]
+    # A drafter path that does not exist would load the target and decode at
+    # the autoregressive rate, which reads as "the new runtime is slow".
+    [ -f "$(argv_value --dflash-model)" ]
+    [ "$(argv_value --model)" != "$(argv_value --dflash-model)" ]
 }

@test "frank-large stays text-only" {
-    # The new model is text-only (no vision projector). The fleet's
-    # image path is frank-small; models.yml declares frank-large text-only.
-    ! grep -q -- '--mmproj' "$SERVE"
+    # The payload has no mmproj, so an --mmproj would point at nothing. The
+    # fleet's image path is frank-small; models.yml declares frank-large
+    # text-only.
+    serve_argv
+    ! argv_has --mmproj
    grep -q -- '--mmproj' "$REPO_ROOT/scripts/serve-frank-small.sh"
 }

-@test "the serve script pins the stock llama.cpp runtime and refuses a missing binary" {
-    grep -q 'unsloth-llama-b11030' "$SERVE"
-    grep -q 'llama-server' "$SERVE"
-    # A missing runtime or payload is a deploy error with a pointer, not a
-    # restart loop.
-    grep -q 'runtime not found' "$SERVE"
-    grep -q 'missing asset' "$SERVE"
-    # The Qwen3.8 template keeps prior thinking blocks across turns only
-    # when the kwarg is set.
-    grep -q 'preserve_thinking' "$SERVE"
-    # The chat template file is wired (not just the kwarg) — the 73k-token
-    # fallback incident (2026-09-21) was a missing --chat-template-file.
-    grep -q 'chat-template-file' "$SERVE"
+@test "the engine is asked for greedy decoding and for preserved prior reasoning" {
+    # Greedy is what this endpoint has always served; a silent switch to
+    # sampled output would be a model-behaviour change disguised as a
+    # runtime swap. Prior thinking blocks must survive across turns or a
+    # multi-turn agent loses its own reasoning between tool calls.
+    serve_argv
+    [ "$(argv_value --temperature)" = "0" ]
+    [ "$(argv_value --preserve-thinking)" = "on" ]
 }

-@test "the serve script wires the DFlash2 spec method and the drafter" {
-    # The new model uses DFlash2 (block-diffusion drafter) at depth 4.
-    grep -q -- '--spec-type draft-dflash' "$SERVE"
-    grep -q -- '--model-draft' "$SERVE"
-    grep -q -- '--spec-draft-n-max' "$SERVE"
-    # The drafter runs on the iGPU (ROCm0).
-    grep -q -- '--spec-draft-device ROCm0' "$SERVE"
+@test "the serve script refuses a missing runtime with the build pointer" {
+    rm -f "$RUNTIME_DIR/bin/gufo"
+    serve_argv
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"runtime not found"* ]]
+    [[ "$output" == *"build-gufo.sh"* ]]
+    # A restart loop would be the alternative; nothing may be exec'd.
+    ! argv_has serve
 }

-@test "the serve script uses q8_0 KV and greedy sampling" {
-    # q8_0 KV — the unsloth b11030 build does not support fp8 KV (allowed:
-    # f32/f16/bf16/q8_0/q4_0/q4_1/iq4_nl/q5_0/q5_1); q8_0 is the
-    # highest-precision 8-bit option and preserves the ~8 GiB footprint.
-    grep -q -- '--cache-type-k q8_0' "$SERVE"
-    grep -q -- '--cache-type-v q8_0' "$SERVE"
-    # Greedy sampling (DFlash2 needs greedy verification).
-    grep -q -- '--temp 0' "$SERVE"
+@test "the serve script refuses a missing drafter with a fetch pointer" {
+    # The drafter lives in its own directory now, so the target can be present
+    # and this still has to be caught.
+    rm -f "$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf"
+    serve_argv
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"missing asset"* ]]
+    [[ "$output" == *"Qwen3.8-27B-DFlash2-Q4_K_M.gguf"* ]]
+    [[ "$output" == *"download-frank-large"* ]]
+    ! argv_has serve
 }

-@test "the payload is fetched resumably and verified against declared sizes + sha256" {
-    grep -q 'unsloth/Qwen3.8-27B-GGUF' "$FETCH"
-    grep -q 'z-lab/Qwen3.8-27B-DFlash2-GGUF' "$FETCH"
-    grep -q -- '-fL -C -' "$FETCH"
-    # Each file's local sha256 is compared to its declared oid and a
-    # mismatch fails the run.
-    grep -q 'SHA256 MISMATCH' "$FETCH"
+@test "the serve script refuses a missing target and names the file" {
+    rm -f "$MODEL_DIR/$TARGET_STUB"
+    serve_argv
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"missing asset"* ]]
+    [[ "$output" == *"$TARGET_STUB"* ]]
 }

-@test "the fleet endpoint and alias survive the cutover" {
-    grep -q '^HOST=0.0.0.0$' "$ENV_FILE"
-    grep -q '^PORT=7770$' "$ENV_FILE"
-    grep -q '^MODEL_DIR=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL$' "$ENV_FILE"
-    grep -q -- '--alias frank-large' "$SERVE"
+@test "both payload files are fetched resumably and verified against declared sizes + sha256" {
+    # The target and the drafter come from DIFFERENT repositories, fetched by
+    # two scripts. Anchor the repo checks to the URL ASSIGNMENT, not to the
+    # file: the same repo string also appears in each script's header comment,
+    # so a plain grep stayed green after the URL was repointed at another
+    # repository -- the script would then download something the serve script
+    # never asked for.
+    grep -qE '^URL="https://huggingface\.co/ajgazin/Swift-1\.5-Qwen3\.8-27B-Uncensored-Dynamic-MTP-GGUF/resolve/main/' "$FETCH"
+    grep -qE 'https://huggingface\.co/z-lab/Qwen3\.8-27B-DFlash2-GGUF/resolve/main/' "$DRAFTER_FETCH"
+    for f in "$FETCH" "$DRAFTER_FETCH"; do
+        # -f so an HTTP error is a failure rather than a small "artifact";
+        # -C - so an interrupted 29 GiB run resumes.
+        grep -q -- '-fL -C -' "$f"
+        # The size and the sha256 are compared after the transfer, not trusted.
+        grep -q 'SHA256 MISMATCH' "$f"
+        grep -q 'SIZE MISMATCH' "$f"
+    done
+}
+
+@test "the serve script and the fetch scripts agree on the payload filenames" {
+    # The serve script's MODEL_FILE and the fetch script's REL are two
+    # independent spellings of one filename, and a repo is not required to
+    # keep them in step: a re-upload under a new name, or an env edited
+    # without the fetch, leaves the endpoint unable to start with a
+    # "missing asset" that no test would have predicted. Bind them here.
+    local served fetched drafter_served drafter_fetched
+    served="$(basename "$(grep '^MODEL_FILE=' "$ENV_FILE" | cut -d= -f2-)")"
+    fetched="$(sed -n 's/^REL="\(.*\)"$/\1/p' "$FETCH")"
+    [ -n "$served" ]
+    [ "$served" = "$fetched" ]
+    drafter_served="$(basename "$(grep '^DRAFTER_FILE=' "$ENV_FILE" | cut -d= -f2-)")"
+    drafter_fetched="$(sed -n 's/^ *"\(Qwen3\.8-27B-DFlash2[^ ]*\.gguf\) .*/\1/p' "$DRAFTER_FETCH")"
+    [ -n "$drafter_served" ]
+    [ "$drafter_served" = "$drafter_fetched" ]
+}
+
+@test "the drafter is still fetched by the script that carries its Hub repo" {
+    # Guards a future "simplification" that folds the drafter into the Swift
+    # script and silently orphans the artifact the serve script still points
+    # at via DRAFTER_DIR.
+    grep -q 'Qwen3.8-27B-UD-Q8_K_XL' "$DRAFTER_FETCH"
+    ! grep -q 'DFlash2' "$FETCH"
+}
+
+@test "the build script refuses a source tree that is not the pinned commit" {
+    # gufo ships no prebuilt Linux binaries, so the runtime is built here.
+    # A tag that moves under us would silently install different kernels
+    # under the same RUNTIME_DIR, so the pin is enforced, not documented.
+    local src="$STAGE/not-the-pinned-commit"
+    mkdir -p "$src"
+    git -C "$src" init -q
+    git -C "$src" -c user.email=t@t -c user.name=t commit -q --allow-empty -m drift
+    run env GUFO_SRC_DIR="$src" GUFO_PREFIX="$STAGE/absent-prefix" bash "$BUILD" --verify
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"GUFO_COMMIT mismatch"* ]]
+}
+
+@test "the build script reports a runtime that was never built" {
+    # --verify on a host with no install must say so rather than exit 0 on
+    # an empty check.
+    run env GUFO_SRC_DIR="$STAGE/no-source-tree" GUFO_PREFIX="$STAGE/absent-prefix" bash "$BUILD" --verify
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"no gufo at"* ]]
+}
+
+@test "the deploy handler gates on the runtime, the payload and the port" {
+    grep -q 'relink_unit frank-large' "$REMOTE_DEPLOY"
+    grep -q 'systemctl enable frank-large.service' "$REMOTE_DEPLOY"
+    grep -q 'wait_port_free 7770' "$REMOTE_DEPLOY"
+    grep -q '127.0.0.1:7770/health' "$REMOTE_DEPLOY"
+    grep -q 'LLM_LARGE_RUNTIME' "$REMOTE_DEPLOY"
+    grep -q 'LLM_LARGE_MODEL' "$REMOTE_DEPLOY"
+    grep -q 'LLM_LARGE_DRAFT' "$REMOTE_DEPLOY"
+}
+
+@test "the deploy handler will not stop a serving endpoint it cannot replace" {
+    # The runtime gate must be live (a real conditional on an executable
+    # path, not a branch that never fires) and it must return BEFORE the
+    # stop, or a host whose gufo was never built takes frank-large down on
+    # its way to reporting the failure.
+    local large gate bail stop
+    large="$(sed -n '/^deploy_llm_large() {/,/^}/p' "$REMOTE_DEPLOY")"
+    [[ "$large" == *'build-gufo.sh'* ]]
+    gate="$(grep -n 'if \[\[ ! -x "\$LLM_LARGE_RUNTIME" \]\]' <<<"$large" | head -1 | cut -d: -f1)"
+    stop="$(grep -n 'systemctl stop frank-large' <<<"$large" | head -1 | cut -d: -f1)"
+    [ -n "$gate" ]
+    [ -n "$stop" ]
+    bail="$(awk -v g="$gate" 'NR > g && /return 1/ { print NR; exit }' <<<"$large")"
+    [ -n "$bail" ]
+    [ "$bail" -lt "$stop" ]
+}
+
+@test "the stray-run sweep keys on the drafter and cannot take frank-small down" {
+    # The sweep used to key on the TARGET filename, which the 2026-09-28
+    # payload swap moved to a different repository. It would have stopped
+    # matching the live process, and a stray run would then hold 7770 past the
+    # port gate and fail the deploy. The drafter is the part that did NOT
+    # move, and every frank-large command line carries it via --dflash-model.
+    grep -q "pkill -f 'Qwen3.8-27B-DFlash2-Q4_K_M'" "$REMOTE_DEPLOY"
+    # frank-small serves a different GGUF on the 3090 with no DFlash2 drafter,
+    # so a binary-name sweep is the only thing that would endanger it.
+    local large
+    large="$(sed -n '/^deploy_llm_large() {/,/^}/p' "$REMOTE_DEPLOY")"
+    ! grep -q 'pkill -x llama-server' <<<"$large"
+    ! grep -q 'pkill -x gufo' <<<"$large"
 }

@test "the deploy handler retires the halogen quadlet without deleting it" {
    grep -q 'retire_halogen_large_quadlet' "$REMOTE_DEPLOY"
    grep -q 'HOME/.AGENTTRASH' "$REMOTE_DEPLOY"
    ! grep -qE '(^|[^[:alnum:]_])rm (-[a-zA-Z]+ )?.*frank-large' "$REMOTE_DEPLOY"
-}
-
-@test "the deploy handler installs the root unit, gates the port and refuses a missing payload" {
-    grep -q 'relink_unit frank-large' "$REMOTE_DEPLOY"
-    grep -q 'systemctl enable frank-large.service' "$REMOTE_DEPLOY"
-    grep -q 'wait_port_free 7770' "$REMOTE_DEPLOY"
-    grep -q '127.0.0.1:7770/health' "$REMOTE_DEPLOY"
-    grep -q 'LLM_LARGE_MODEL' "$REMOTE_DEPLOY"
-    grep -q 'LLM_LARGE_DRAFT' "$REMOTE_DEPLOY"
-}
-
-@test "the stray-run sweep cannot take frank-small down" {
-    # frank-small on this host is ALSO a llama-server. The sweep must match
-    # the model path, not the binary name, or a frank-large deploy restarts
-    # the 3090 endpoint.
-    grep -q "pkill -f 'Qwen3.8-27B-UD-Q8_K_XL'" "$REMOTE_DEPLOY"
-    # ...and only inside the large handler: extract it and check there is no
-    # bare binary-name pkill in it.
-    local large
-    large="$(sed -n '/^deploy_llm_large() {/,/^}/p' "$REMOTE_DEPLOY")"
-    ! grep -q 'pkill -x llama-server' <<<"$large"
-}
-
-@test "the halogen quadlet is archived, not live" {
+    # Nothing live ships for this endpoint as a container any more. The
+    # archived copy is host state in an ignored .AGENTTRASH — deliberately
+    # untracked, and present only in a primary checkout — so search the
+    # tracked tree, not the filesystem.
    [ ! -e "$REPO_ROOT/dankclaw/frank-large/quadlet/frank-large.container" ]
-    [ -f "$REPO_ROOT/dankclaw/.AGENTTRASH/frank-large.container.halogen" ]
+    ! git -C "$REPO_ROOT" grep -q 'Image=.*halogen' -- dankclaw
 }
Author	SHA1	Message	Date
Frank	f532e03bf4	docs: GLM-5.3-Flash readiness revised — tcclaviger fork has native MXFP4/DSFP4 Source reading in the qwen-fp8 container: glm5next is registered with an amd/ backend dir + DSA indexer; r4d.so has native moe_dsfp4/moe_dsfp4_mt1 entries (has_dsfp4()==True), so the old 'MXFP4 = emulation, missing kernels' verdict is dead for this fork. DSFP4 = MXFP4 weights + DeepSeek activation law. Offload recipes exist for mxfp4 and W4A16 (R4dW4a16MoEExperts in offloader/expert.py:114). GLM gate is now the payload format (compressed-tensors MXFP4/W4A16, none public) + a HIP boot probe.	2026-09-29 23:28:04 -07:00
Frank	57072a44df	docs: final 256k verdict — boots at 4.03-4.5 GiB KV, band 34.1-36.5 straddles target The KV trim is inert: arena identical (5,648 slots) at 4.5 and 4.03 GiB (planner buckets; the trim went to margin). Final 256k config MAXLEN=262144 KV_BYTES=4328521728 prefetch off; full-speed alternative MAXLEN=184224 at 3 GiB.	2026-09-29 21:55:58 -07:00
Frank	0ccb01708b	docs: qwen38-fp8 expert-offload tuning — prefetch off hits 37.4-40.6 tok/s (target met) Measured campaign on the ad hoc qwen-fp8 container (tcclaviger RDNA4 fork, 2x R9700, TP=2): telemetry (with_stack=false) measured the prefetcher at 36% precision — ~half of gather traffic wasted; deleting --expert-prefetch: 26.8 -> 40.57/37.44/37.57 tok/s. Prefetch-on shape survived every other challenger (gather geometry, spec depth, TunableOp [corrupts numerics], MNBT, concurrency, ngram [PLE fork bug]). 256k ctx boots at 4.5 GiB KV (36.65/34.21/34.77, arena -14%). New runbook + index links + log entry.	2026-09-29 21:22:21 -07:00
Frank	67672dc794	docs(llm): record a measured NEGATIVE tuning result for frank-large Asked for iterative decode/prefill improvement at a 32k working point without touching the model or the quantization. There isn't one: gufo's served settings are already at this engine's tuned point, so no config change was made and none is proposed. Measured on the repo's own prose and shell sources at 32,762 prompt tokens, salted per run against a warm prefix cache, 3-5 repetitions: * --prefill-chunk 512 -> 8192 moves prefill 410.7-412.4 t/s. A 16x change in chunk size, 0.4% out. The lever is dead at this depth. * --draft-tokens 5 -> 10 gives 13.79-15.86 t/s decode, all inside the run-to-run noise. The 5-rep confirmation REVERSED the 3-rep leaders: dt9+chunk4096 looked best and came last (12.86), the deployed config came first (14.71). * --draft-policy fixed vs adaptive: no effect. --min-draft-tokens: rejected by the engine ("DFlash2 requires ... 1"). MTP: unsupported for Q8 targets (token_embd.weight is Q8_0). * --context 262144 -> 40960, tested as a diagnostic: no gain, and it would break models.yml's declared contextWindow. Not shipped. What does move the number is speculation itself — 6.65 t/s off vs ~15 t/s on, 5.3x — and it is already enabled. The two traps that cost most of the wall time are now in AGENTS.md's benchmarking policy, because both produced FALSE results: 1. gufo bench cannot rank speculative settings and inverted the ranking. Its corpus reports acceptance=0.933 +/- 0.000 with an identical trace sha256 across draft widths — the same highly-predictable text every run. It reported 29-37 t/s where real traffic gives ~15, and made dt16 look 6% faster than dt7 when real text made it 14% slower. 2. A live A/B needs >=128 generated tokens. At 8, inter_token is first-token noise; the first pass produced a nonsense ordering that the 128-token pass reversed. Plus the Server-Timing header is `ttft;dur=402.485` — the unit rides in the NAME half, so splitting on "=" keys the dict "ttft;dur" and every lookup silently returns nothing. Four configs reported "no Server-Timing" while the header was present on all of them.	2026-09-29 05:06:06 -07:00
Frank	3a931ce2c1	chore(dankclaw): reclaim 510G of retired payloads; fix a stale fallback The disk was at 93% with 137G free. Nine retired payloads are now moved to ~/.AGENTTRASH/20260928-retired-models/ with a MANIFEST.md giving a restore command per item — moved, not deleted, and nothing is freed until the purge is approved. Every one maps to a runtime already retired in the runbook index. Only the .gguf was taken from Qwen3.8-27B-UD-Q8_K_XL/: the live frank-large DFlash2 drafter lives in that directory and stays put. The manifest warns against restoring the directory wholesale. Kept: checkpoints/ (23G) — unreferenced by the repo, but ComfyUI is live and its workflows were not searched, so it is not confirmed dead. Also logs/, because x4e-run.log is a SPARSE file (3.3G apparent, 4K real) and frees nothing. The reference check caught a live bug: serve-frank-small.sh's fallback MODEL_DIR still pointed at the retired GSQ-RCO payload frank-small served on 2026-09-16/17. frank-small.env has always overridden it, so the drift was invisible — but a missing or unreadable env file would have turned a config problem into a "missing asset" naming a model that is no longer there. Repointed at Gemma4-12B-QAT/, which the env file already sets. Not taken, by request: podman images (279G reclaimable, 95%) and the uv+pip caches (73G). Together 352G of regenerable garbage, larger than what was reclaimed here. The 29.3G payload rollback target went with the rest, so the payload-only rollback is now a re-fetch first; the runbook and AGENTS.md say so.	2026-09-29 00:00:49 -07:00
Frank	7f9424554c	frank-large: serve the abliterated Swift-1.5 UD-Q8_K_XL The target moves from unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL to ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF's UD-Q8_K_XL — an abliterated fine-tune of the same base, under the non-OSI SWIFT OPEN LICENCE 1.0. Port, alias, context, drafter and client contract are unchanged. Compatibility was settled from the GGUF headers before fetching anything: both files parse to the same architecture (qwen35), the same 866 tensors with identical names in identical order, zero per-tensor ggml_type or dim mismatches, and the same MTP head. Two metadata differences that look alarming are no-ops, confirmed against gufo's source — general.file_type is deliberately not consulted (the file gufo already served records Q4_K_M while containing no Q4_K tensor), and Swift's extra qwen35.attention.recurrent_layers key is never read; layer kinds come from full_attention_interval, which both files carry as 4 and which agrees. The load test agreed: 9.2 s to listening, 66.5 GiB of the 116.7 GiB iGPU, no loader warnings, and a green contract battery including a two-turn tool loop. The DFlash2 drafter did not move, so the payload is now two files in two directories: DRAFTER_DIR is its own tunable. That broke the deployer's stray-run sweep, which keyed on the target filename and would have stopped matching — a stray run would then have held 7770 past the port gate. It now keys on the drafter, which every frank-large command line carries. The fetch script sends no auth header: the repo is public, and a token on the curl command line is world-readable via /proc/<pid>/cmdline. The anonymous transfer ran at 1.8 MiB/s; the host token through a mode-600 curl -K config reached 7 MiB/s. Test seam: the mutation pass found four real gaps. Two fetch assertions were satisfied by a header comment rather than the URL assignment, so repointing a repository left them green, and nothing bound the env's MODEL_FILE/DRAFTER_FILE to the filenames the fetch scripts download. Both are now explicit. A harness bug worth recording: a bats --filter that matches no test reports zero "not ok", indistinguishable from a mutation that survived — the harness now checks the 1..N count. 16/16 caught.	2026-09-28 23:29:05 -07:00
Frank	32abc8d543	chore(llm): add dankstatus.sh — one verified parse path for Dankserver state The ad-hoc status commands kept producing wrong readings, most seriously reporting "MemAvailable 0.0 GiB" on a host that actually had 240 GiB free. Cause: /proc/meminfo separates key from value with a TAB, so grep MemAvailable /proc/meminfo \| tr -s " " \| cut -d" " -f2 returns an empty field and awk prints 0.0. A false "host is out of memory" reading is worse than no reading, because it invites an unnecessary abort. dankstatus.sh parses on a colon split (tab-agnostic), cross-checks MemAvailable against MemTotal before printing either, and refuses to print a zero value. It also pins the rocm-smi CSV column order (device, total, used) which was previously read swapped, and takes only the first line of systemctl is-active/is-enabled, which emit more than one line here. Incidental finding while validating: rocm-smi reports card2 (the 7700X iGPU) as 0.50 GiB total, not the 123.0 GiB torch.cuda reports. torch was exposing the GTT aperture. The device is smaller than documented, not larger. Tests: 346/346 pass (no test surface touched).	2026-09-28 23:27:50 -07:00
Frank	b4b6777e49	fetch: add the Swift-1.5 frank-large candidate download Public, ungated repo so no auth header is sent — a token on the curl command line is world-readable via /proc/<pid>/cmdline. Size and sha256 are the Hub LFS values (31457990272 / 9076f99a…), verified after fetch. Not wired into frank-large.env: the gufo load test has to pass first.	2026-09-28 22:48:53 -07:00
Frank	9d9d28cda6	docs(llm): record the gufo cutover verification Live numbers from the 2026-09-28 deploy: 13 s load to listening, RSS 30.6 GiB, ttft 549.8 ms / inter-token 24.2 ms on a short prompt, and a green contract battery (health, ready, /v1/models at 262,144, content + reasoning_content, tool_calls, SSE, and a two-turn tool loop). Also recorded: gufo accepts every request field the harness sends in extraBody and the full reasoning_effort set, both verified 200 — which is why models.yml's reasoningEffortMap fold is now optional rather than required, and why it is being kept anyway.	2026-09-28 22:28:44 -07:00
Frank	a196b0fa45	test: scope the halogen-quadlet check to tracked files The .AGENTTRASH archive is ignored and exists only in a primary checkout, so a filesystem grep fails there while passing in a worktree. git grep searches the tracked tree, which is what the assertion is about.	2026-09-28 22:12:16 -07:00
Frank	aaef419a6d	frank-large: replace stock llama.cpp with gufo (gufo-org/gufo) on Dankclaw gufo is a vertical inference engine built for exactly this hardware (Ryzen AI MAX+ 395 / Radeon 8060S, gfx1151), and Dankclaw's frank-large has swapped runtimes seven times in search of a better one. This one ships no prebuilt Linux binary, so the runtime is BUILT FROM SOURCE on the host and the payload, port, alias and client contract stay exactly where they are. Runtime: gufo v0.1.1 (b0f8673) at /home/dank/GitPackages/gufo-v0.1.1, built by the new scripts/build-gufo.sh. It links the host ROCm 7.2.4 through its RPATH — the opposite of the unsloth prebuilt it replaces, which carried its own — so two host workarounds are encoded in that script: Ubuntu's librocwmma-dev installs a truncated rocWMMA header set that shadows AMD's and cannot be overridden with -I in HIP mode, and the ROCm clang prefers Ubuntu 26.04's GCC 16 headers over the GCC 15 whose libstdc++ is actually installed. Shape is unchanged: one session, full 262144 ctx, the same UD-Q8_K_XL target and the same DFlash2 drafter, greedy, text-only. Speculation moves from llama.cpp's fixed depth 4 to gufo's adaptive block policy capped at 7 proposals. Client-contract deltas, all documented: the chat template is compiled into the binary (no --chat-template-file, so the Qwen-Sharp 73k-token failure mode is gone), reasoning_effort now takes the full OpenAI set (so models.yml's reasoningEffortMap fold is no longer required — kept anyway, so this commit does not also move every high role from xhigh to high effort), and there is no KV-quant or prompt-cache-RAM flag. The test seam now runs the serve script against a stub engine and asserts on the argv it actually execs, instead of grepping the script's text. The deploy handler gates on the runtime binary and returns before it stops the serving unit. Measured trade, from gufo's own Q8_K_XL tables against llama.cpp b11069: prefill +37% to +80%, cold load 4.5x faster, decode +7% at zero depth — but -14% at 65,536 tokens of depth and -24% at 131,072. On the 8-bit target the crossover is ~32k, and frank-large runs one 262,144 conversation. The Q4_K_XL target shows no such regression.	2026-09-28 22:08:29 -07:00
Frank	d29d104f01	docs: frontier-MoE feasibility on Dankserver (V4.1 / MiMo / GLM-5.3) Answers Frank's question: given the 2x R9700 box, which of DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on what runtime, and can Affinity / vllm-radiance / FreeToken-AMD be modified to run them fast. Sizing, measured from the HF API rather than the model cards: - DS4.1 payload 475.27 GiB, of which 189.13 GiB is Engram (two 94.56 GiB shards, the layer 1 and 14 lookup tables). Lucebox never maps them, so they do not evict the expert page cache. - 240 routed experts x ~11 MB per token at the 37-of-384-per-layer residency the hot stack can hold = ~2.4 GB across PCIe per token. At 55 GB/s real on Gen5 x8 that is a ~23 tok/s ceiling, under the 35 tok/s goal before any attention or dense work. - The backend refuses layer-major prefill for V4.1, so the only legal prefill is token-by-token; 1000 t/s is ~20x away. Affinity (Codeberg StillDeadcode) builds clean here (hipcc 23, gfx1201, zero errors) and claims 110 tok/s for the 284B V4-Flash, because the 101 GiB container leaves the cold expert set in a pinned host pool in RAM. V4.1 is 2.5x too big to repeat that. Port scope, from both checkpoints' real safetensors indices: expert machinery ports unchanged (identical tensor naming, 15,360 tensors at 40x384), 26/32 per-layer tensors and 34/35 config keys present. Five gaps - Engram, the disjoint {0,1,2} vs {0,4,128} ratio vocabulary, CED, the DSpark draft geometry, vision. num_hash_layers is the only key V4.1 lacks, so model.h:36's default of 3 silently survives and would route layers 0-2 through a table that does not exist. Also corrects: Dankserver RAM 60 -> 246 GiB (stale since the 2026-09-20 board swap) and storage; the top-level index credited V4.1 with V4's geometry (43 layers / 256 experts vs the actual 40 / 384).	2026-09-28 14:58:59 -07:00
Frank	564a781539	log: frank-primary deploy-state reconciled to `ba2100e` (autoresearch already live, no restart); e2e 32/33 (Mac mini offline)	2026-09-28 13:22:14 -07:00
Frank	ba2100e22b	docs: prefix caching deployed on frank-primary (E8-qualified, 9x/63x reuse TTFT)	2026-09-28 12:33:28 -07:00
Frank	41e918929f	frank-primary: enable prefix caching (E8 qualified: 9x-63x reuse TTFT, zero VRAM cost, cold path unchanged)	2026-09-28 12:03:56 -07:00
Frank	92579b0674	log: frank-primary restore verified (`f187165` deploy, ~11:00); e2e 32/33, 11-admin blocked by asleep Mac mini	2026-09-28 11:21:43 -07:00