gufo is a vertical inference engine built for exactly this hardware
(Ryzen AI MAX+ 395 / Radeon 8060S, gfx1151), and Dankclaw's frank-large
has swapped runtimes seven times in search of a better one. This one
ships no prebuilt Linux binary, so the runtime is BUILT FROM SOURCE on
the host and the payload, port, alias and client contract stay exactly
where they are.
Runtime: gufo v0.1.1 (b0f8673) at /home/dank/GitPackages/gufo-v0.1.1,
built by the new scripts/build-gufo.sh. It links the host ROCm 7.2.4
through its RPATH — the opposite of the unsloth prebuilt it replaces,
which carried its own — so two host workarounds are encoded in that
script: Ubuntu's librocwmma-dev installs a truncated rocWMMA header set
that shadows AMD's and cannot be overridden with -I in HIP mode, and the
ROCm clang prefers Ubuntu 26.04's GCC 16 headers over the GCC 15 whose
libstdc++ is actually installed.
Shape is unchanged: one session, full 262144 ctx, the same UD-Q8_K_XL
target and the same DFlash2 drafter, greedy, text-only. Speculation
moves from llama.cpp's fixed depth 4 to gufo's adaptive block policy
capped at 7 proposals.
Client-contract deltas, all documented: the chat template is compiled
into the binary (no --chat-template-file, so the Qwen-Sharp 73k-token
failure mode is gone), reasoning_effort now takes the full OpenAI set
(so models.yml's reasoningEffortMap fold is no longer required — kept
anyway, so this commit does not also move every high role from xhigh to
high effort), and there is no KV-quant or prompt-cache-RAM flag.
The test seam now runs the serve script against a stub engine and
asserts on the argv it actually execs, instead of grepping the script's
text. The deploy handler gates on the runtime binary and returns before
it stops the serving unit.
Measured trade, from gufo's own Q8_K_XL tables against llama.cpp
b11069: prefill +37% to +80%, cold load 4.5x faster, decode +7% at zero
depth — but -14% at 65,536 tokens of depth and -24% at 131,072. On the
8-bit target the crossover is ~32k, and frank-large runs one 262,144
conversation. The Q4_K_XL target shows no such regression.
Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference.
llm
2026-09-03
Dankclaw System Specs
Server Overview
Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool.
Hardware Specifications
CPU
Property
Value
Model
AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
Architecture
x86_64
Cores
16 (1 socket, 1 thread per core)
Max MHz
5187.50
Min MHz
625.00
NUMA Nodes
1 (node 0, 31213 MB)
GPU (Integrated)
Property
Value
Model
AMD Radeon 8060S (Strix Halo)
Device ID
0x1586
Architecture
gfx1151 (Strix Halo)
GPU-Visible Memory
~114 GiB GTT aperture (the budget; gtt_total; rocm-smi's smaller SVM carve-out is not the budget)
The NPU is retired (2026-09-11). The frank-small name now serves Gemma4-12B QAT on the RTX 3090 via llama.cpp (see frank-small-gemma4-12b).
ROCm
Property
Value
ROCk Module
Loaded
Runtime Version
1.18
ROCm Version
7.2.4 system (/opt/rocm) + TheRock 7.15 SDK in the CIRU venv (used for the runtime)
Agent Name
AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
GPU Arch
gfx1151 (Strix Halo)
GPU Global Memory
~114 GiB GTT aperture (the budget)
Memory
Property
Value
System RAM
64 GiB (shared CPU + GPU; 61 GiB visible)
GPU-Visible (ROCm)
~114 GiB GTT aperture (the budget)
Swap
8.0 GiB
/dev/shm
~15 GiB (half of RAM)
Storage
Filesystem
Size
Used
Avail
Use%
/ (nvme0n1p2)
1.9T
1.3T
545G
70%
Network
Interface
Type
Speed
IP
Notes
eno1
onboard
2500 Mb/s (2.5 GbE)
192.168.0.143 (DHCP)
Carries the RPC path to Dankserver since 2026-08-17. Default route via 192.168.0.1.
enx00e04cc2e1e5
USB-C to Ethernet adapter
1000 Mb/s
192.168.0.116
UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See Pipeline Parallelism/Ethernet Bandwidth Math section 0.
docker0
bridge
-
172.17.0.1
Usually down.
The L2 gateway/switch at 192.168.0.1 (MAC f0:4b:8a:65:c5:40) is at
least 2.5 GbE-capable. Whether it is 10G-capable is unverified — check
the device itself before buying 10G USB-C adapters; otherwise a direct
cable between claw and server is required for 10G.
Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
The GPU budget is the GTT aperture: ~114 GiB (gtt_total in /sys/class/drm/card1/device/; llama-server --list-devices reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is not the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use free or dmidecode for the GPU budget.
The active service is frank-large on port 7770: gufo (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source at /home/dank/GitPackages/gufo-v0.1.1/, serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, DFlash2 drafter). See frank-large-gufo-qwen38-27b.
The active runtime links against the host ROCm (/opt/rocm → /etc/alternatives/rocm → 7.2.4) through its RPATH, so it needs the host HIP stack rather than a bundled one — the opposite of the retired llama.cpp prebuilt, which carried its own. Building it needs the ROCm dev packages that are installed, and two workarounds recorded in the runbook (drop librocwmma-dev; pin --gcc-install-dir to GCC 15). The retired CIRU venv's TheRock 7.15 SDK (LD_LIBRARY_PATH) is not in the serving path.
The retired vLLM venvs (/home/dank/vllm-venv, /home/dank/vllm-vk-venv) were moved to /home/dank/.AGENTTRASH/ on 2026-08-18.
1.1 TB free disk space is ample for model storage.
Frank
aaef419a6d