frank-large: ROCmFPX llama.cpp on the Q8_0_ROCMFPX_AGENT file, MTP drafting
The repo's highest-quality rung is ftype 115: 311 tensors of GGML_TYPE_Q8_0_ROCMFPX, a type only the ROCmFPX fork reads, so the runtime moves from gufo to scripts/build-rocmfpx.sh's pinned fork build (36e9acd) — the first build on this host measured to load this file without the 2026-08-25 set_rows segfault. One session at the native 262,144 window, f16 KV (64 KiB/token, 16 GiB), the repo's Q4_0 MTP head drafting 4 tokens, plus the bundled BF16 projector so the endpoint has vision again. The serve script now honors caller environment overrides over the env file (a plain source let the file win, which made single-run overrides no-ops) and defaults apply only when a value is unset, so an explicit LLAMA_BIN='' is the error it reads as.
This commit is contained in:
1 parent
d82c5c03b7
commit
125d04967f
6 files changed
+608
-495
No files matched your search
@@ -1,5 +1,5 @@
|
||||
[Unit]
|
||||
Description=gufo (gufo-org/gufo, ROCm gfx1151) Qwen3.8-27B UD-Q8_K_XL, DFlash2 spec decode, 262144 ctx / 1 session, root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
|
||||
Description=ROCmFPX llama.cpp (gfx1151) kingjones777 Qwen3.8-27B Q8_0_ROCMFPX_AGENT, MTP spec decode, 262144 ctx / 1 session, root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
@@ -13,7 +13,9 @@ Group=dank
|
||||
LimitMEMLOCK=infinity
|
||||
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
|
||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
|
||||
Restart=on-failure
|
||||
# always, not on-failure: an engine that drains and exits 0 after a worker
|
||||
# fault would otherwise leave the endpoint down until someone notices.
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
[Install]
|
||||
|
||||
Reference in new issue
Block a user