Frank
aaef419a6d
gufo is a vertical inference engine built for exactly this hardware (Ryzen AI MAX+ 395 / Radeon 8060S, gfx1151), and Dankclaw's frank-large has swapped runtimes seven times in search of a better one. This one ships no prebuilt Linux binary, so the runtime is BUILT FROM SOURCE on the host and the payload, port, alias and client contract stay exactly where they are. Runtime: gufo v0.1.1 (b0f8673) at /home/dank/GitPackages/gufo-v0.1.1, built by the new scripts/build-gufo.sh. It links the host ROCm 7.2.4 through its RPATH — the opposite of the unsloth prebuilt it replaces, which carried its own — so two host workarounds are encoded in that script: Ubuntu's librocwmma-dev installs a truncated rocWMMA header set that shadows AMD's and cannot be overridden with -I in HIP mode, and the ROCm clang prefers Ubuntu 26.04's GCC 16 headers over the GCC 15 whose libstdc++ is actually installed. Shape is unchanged: one session, full 262144 ctx, the same UD-Q8_K_XL target and the same DFlash2 drafter, greedy, text-only. Speculation moves from llama.cpp's fixed depth 4 to gufo's adaptive block policy capped at 7 proposals. Client-contract deltas, all documented: the chat template is compiled into the binary (no --chat-template-file, so the Qwen-Sharp 73k-token failure mode is gone), reasoning_effort now takes the full OpenAI set (so models.yml's reasoningEffortMap fold is no longer required — kept anyway, so this commit does not also move every high role from xhigh to high effort), and there is no KV-quant or prompt-cache-RAM flag. The test seam now runs the serve script against a stub engine and asserts on the argv it actually execs, instead of grepping the script's text. The deploy handler gates on the runtime binary and returns before it stops the serving unit. Measured trade, from gufo's own Q8_K_XL tables against llama.cpp b11069: prefill +37% to +80%, cold load 4.5x faster, decode +7% at zero depth — but -14% at 65,536 tokens of depth and -24% at 131,072. On the 8-bit target the crossover is ~32k, and frank-large runs one 262,144 conversation. The Q4_K_XL target shows no such regression.
97 lines
4.1 KiB
Bash
Executable File
97 lines
4.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Build the gufo runtime that serves frank-large on Dankclaw.
|
|
#
|
|
# bash scripts/build-gufo.sh # build + install the pinned tag
|
|
# bash scripts/build-gufo.sh --verify # only check the installed tree
|
|
#
|
|
# gufo (gufo-org/gufo) is the Strix Halo inference engine: it targets
|
|
# gfx1151 specifically and has no prebuilt Linux release binaries, so the
|
|
# runtime is built from the pinned git tag and installed to
|
|
# /home/dank/GitPackages/gufo-<tag>/. Everything below is what Dankclaw
|
|
# needs on top of AMD's Linux instructions.
|
|
#
|
|
# The host already carries the ROCm 7.2.4 stack (/opt/rocm -> /etc/
|
|
# alternatives/rocm) with the HIP dev packages, so this script installs
|
|
# only the ordinary system libraries gufo links against.
|
|
#
|
|
# Two host facts this build depends on, both discovered on 2026-09-28:
|
|
#
|
|
# 1. Ubuntu's librocwmma-dev ships a TRUNCATED rocWMMA header set (the
|
|
# public headers without rocwmma/internal/) into /usr/include, where
|
|
# it shadows AMD's complete copy under /opt/rocm/include. The first
|
|
# build died on "'internal/accessors.hpp' file not found" from
|
|
# /usr/include/rocwmma/rocwmma.hpp. A -I cannot fix it: in HIP mode
|
|
# the host compilation pass searches the system directories first.
|
|
# The package has no reverse dependencies, so it is removed.
|
|
# 2. The ROCm clang picks the NEWEST /usr/include/c++ tree, which on
|
|
# Ubuntu 26.04 is GCC 16's. Its <atomic_wait.h> emits a symbol
|
|
# reference the installed libstdc++ (GLIBCXX_3.4.35, GCC 15) does not
|
|
# export, and the link fails with "undefined symbol:
|
|
# std::__detail::__notify_impl". --gcc-install-dir pins the headers to
|
|
# the same GCC the runtime library came from.
|
|
set -euo pipefail
|
|
|
|
GUFO_TAG="${GUFO_TAG:-v0.1.1}"
|
|
GUFO_REPO="${GUFO_REPO:-https://github.com/gufo-org/gufo}"
|
|
# The commit the tag resolved to when this runtime was qualified. A tag that
|
|
# moves is an unreproducible runtime, so the pin is checked, not assumed.
|
|
GUFO_COMMIT="${GUFO_COMMIT:-b0f8673470a3dcc2b5e9c9546c48a40b82b5ff31}"
|
|
SRC_DIR="${GUFO_SRC_DIR:-/home/dank/GitPackages/gufo-src}"
|
|
PREFIX="${GUFO_PREFIX:-/home/dank/GitPackages/gufo-${GUFO_TAG}}"
|
|
JOBS="${JOBS:-$(nproc)}"
|
|
# The GCC toolchain that owns the installed libstdc++ on this host.
|
|
GCC_DIR="${GCC_DIR:-/usr/lib/gcc/x86_64-linux-gnu/15}"
|
|
|
|
# check_pin — refuse a source tree that is not the qualified commit.
|
|
check_pin() {
|
|
local commit
|
|
commit="$(git -C "$SRC_DIR" rev-parse HEAD 2>/dev/null || true)"
|
|
if [[ -n "$commit" && "$commit" != "$GUFO_COMMIT" ]]; then
|
|
echo "GUFO_COMMIT mismatch: $SRC_DIR is at $commit, pinned $GUFO_COMMIT" >&2
|
|
exit 1
|
|
fi
|
|
}
|
|
|
|
if [[ "${1:-}" == "--verify" ]]; then
|
|
# The pin first: a moved tag is the more urgent thing to hear about, and
|
|
# it is the check that must not be masked by a missing install.
|
|
check_pin
|
|
[[ -x "$PREFIX/bin/gufo" ]] || { echo "no gufo at $PREFIX/bin/gufo" >&2; exit 1; }
|
|
"$PREFIX/bin/gufo" diagnose | tail -n 5
|
|
exit 0
|
|
fi
|
|
|
|
echo "== system libraries gufo links against =="
|
|
sudo -n apt-get install -y -qq --no-install-recommends \
|
|
build-essential cmake ninja-build pkg-config \
|
|
libicu-dev libcurl4-openssl-dev libssl-dev libpng-dev libjpeg-dev ffmpeg
|
|
|
|
echo "== dropping Ubuntu's truncated rocWMMA headers =="
|
|
if dpkg -s librocwmma-dev >/dev/null 2>&1; then
|
|
sudo -n apt-get remove -y -qq librocwmma-dev
|
|
fi
|
|
|
|
echo "== source: $GUFO_REPO @ $GUFO_TAG =="
|
|
if [[ ! -d "$SRC_DIR/.git" ]]; then
|
|
mkdir -p "$(dirname "$SRC_DIR")"
|
|
git clone --depth 1 --branch "$GUFO_TAG" "$GUFO_REPO" "$SRC_DIR"
|
|
fi
|
|
git -C "$SRC_DIR" fetch --depth 1 origin "refs/tags/$GUFO_TAG"
|
|
git -C "$SRC_DIR" checkout -q --detach "refs/tags/$GUFO_TAG"
|
|
echo " $(git -C "$SRC_DIR" rev-parse HEAD)"
|
|
check_pin
|
|
|
|
echo "== configure + build (release, HIP gfx1151) =="
|
|
cd "$SRC_DIR"
|
|
toolchain="--gcc-install-dir=$GCC_DIR"
|
|
cmake --preset release \
|
|
-DCMAKE_INSTALL_PREFIX="$PREFIX" \
|
|
-DCMAKE_HIP_FLAGS="$toolchain" \
|
|
-DCMAKE_EXE_LINKER_FLAGS="$toolchain"
|
|
cmake --build --preset release --parallel "$JOBS"
|
|
cmake --install build/release
|
|
|
|
echo "== verify =="
|
|
"$PREFIX/bin/gufo" diagnose
|
|
echo "frank-large runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"
|