frank-large: ROCmFPX llama.cpp on the Q8_0_ROCMFPX_AGENT file, MTP drafting

The repo's highest-quality rung is ftype 115: 311 tensors of
GGML_TYPE_Q8_0_ROCMFPX, a type only the ROCmFPX fork reads, so the runtime
moves from gufo to scripts/build-rocmfpx.sh's pinned fork build (36e9acd) —
the first build on this host measured to load this file without the
2026-08-25 set_rows segfault. One session at the native 262,144 window, f16
KV (64 KiB/token, 16 GiB), the repo's Q4_0 MTP head drafting 4 tokens, plus
the bundled BF16 projector so the endpoint has vision again.

The serve script now honors caller environment overrides over the env file
(a plain source let the file win, which made single-run overrides no-ops) and
defaults apply only when a value is unset, so an explicit LLAMA_BIN='' is the
error it reads as.
This commit is contained in:
Frank committed 2026-10-02 00:47:16 -07:00
1 parent d82c5c03b7
commit 125d04967f
6 files changed
+608 -495

No files matched your search

+4 -2
View File
@@ -1,5 +1,5 @@
[Unit]
Description=gufo (gufo-org/gufo, ROCm gfx1151) Qwen3.8-27B UD-Q8_K_XL, DFlash2 spec decode, 262144 ctx / 1 session, root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
Description=ROCmFPX llama.cpp (gfx1151) kingjones777 Qwen3.8-27B Q8_0_ROCMFPX_AGENT, MTP spec decode, 262144 ctx / 1 session, root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
After=network-online.target
Wants=network-online.target
@@ -13,7 +13,9 @@ Group=dank
LimitMEMLOCK=infinity
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
Restart=on-failure
# always, not on-failure: an engine that drains and exits 0 after a worker
# fault would otherwise leave the endpoint down until someone notices.
Restart=always
RestartSec=10
[Install]