dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-large: swap to the uncensored Q6_0_ROCMFPX_AGENT payload

Browse Source 
Frank's order: replace frank-large with
kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF's
Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf, following that card.

The file is ftype 114 — 339 GGML_TYPE_Q6_0_ROCMFPX (102) + 159
GGML_TYPE_Q8_0_ROCMFPX (103) + 353 F32 over 851 tensors,
qwen35/64 layers/full_attention_interval 4/4 KV heads/262,144 window —
read out of its own header. The pinned ROCmFPX build already knows both
types (ggml.h 102/103 plus convert/cpy/getrows/mmq/fattn device paths
for 102), so no rebuild: the fork stays at 36e9acd and the payload
directory is the only new host state.

Card compliance, all already our flags: Q4_0 draft head (not Q8_0),
--spec-draft-n-max 4 (the knee on both cards), --spec-draft-ngl 99,
--jinja, --fit off, -fa on, -dio, one slot at full window, thinking with
the card's own sampling triple (which the file also embeds as its
general.sampling defaults). Kept from the deployed shape: the BF16
projector (this repo ships only a Q8_0 one; BF16 is the file verified
4/4 on spatial ground truth here) and the byte-identical MTP head, which
the new script hardlinks instead of re-fetching 1.6 GiB.

Franksdank-Runbook: serverAppConfigs-docs/LLM/frank-large-rocmfpx-q6agent-uncensored-mtp.md
This commit is contained in:
Frank
2026-10-03 02:04:22 -07:00
parent 0ae923b879
commit 7d3dd33de0
5 changed files with 236 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/deploy-remote.sh
+21 -18
View File
@@ -1088,27 +1088,30 @@ deploy_anubis() {
# frank-large is the ROCmFPX fork of llama.cpp (kingjones30/ROCmFPX, pinned by
# scripts/build-rocmfpx.sh), serving
# kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF:Q8_0_ROCMFPX_AGENT with the
# repo's own Q4_0 MTP draft head and its BF16 vision projector, on the Strix
# Halo iGPU. It is a ROOT unit: the runtime is a native binary linked against
# the host ROCm, so there is no container image to install and no generated
# unit to dodge. relink_unit points /etc/systemd/system/frank-large.service at
# the repo file and boot persistence is that unit's [Install] section.
# kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF:Q6_0_ROCMFPX_AGENT
# with the repo's own Q4_0 MTP draft head and the BF16 vision projector, on the
# Strix Halo iGPU. It is a ROOT unit: the runtime is a native binary linked
# against the host ROCm, so there is no container image to install and no
# generated unit to dodge. relink_unit points
# /etc/systemd/system/frank-large.service at the repo file and boot persistence
# is that unit's [Install] section.
#
# The runtime gate matters as much as the payload gate: the fork is built from
# source on this host, and the payload is ftype 115 (tensor type 103,
# GGML_TYPE_Q8_0_ROCMFPX) which only this fork can read — so a machine whose
# /home/dank/GitPackages tree was never built has a unit that can never
# become healthy, and a green deploy would leave the endpoint dark.
# source on this host, and the payload is ftype 114 (tensor types 102
# GGML_TYPE_Q6_0_ROCMFPX and 103 GGML_TYPE_Q8_0_ROCMFPX) which only this fork
# can read — so a machine whose /home/dank/GitPackages tree was never built has
# a unit that can never become healthy, and a green deploy would leave the
# endpoint dark.
#
# It replaces gufo v0.1.1 + unsloth UD-Q4_K_XL (the rollback target, still
# built by scripts/build-gufo.sh with its payload on disk), which had replaced
# the stock llama.cpp prebuilt. The port gate is the same as every cutover
# It replaces the ALIGNED kingjones Q8_0_ROCMFPX_AGENT file (rollback target:
# its payload and its draft head are still on disk in
# ~/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP, where this payload's
# draft-head hardlink also lives). The port gate is the same as every cutover
# since 2026-09-16: nothing starts until 7770 is released.
LLM_LARGE_RUNTIME="${LLM_LARGE_RUNTIME:-/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server}"
LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf}"
LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mtp-Qwen3.8-27B-Q4_0.gguf}"
LLM_LARGE_MMPROJ="${LLM_LARGE_MMPROJ:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mmproj-Qwen3.8-27B-BF16.gguf}"
LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf}"
LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/mtp-Qwen3.8-27B-Q4_0.gguf}"
LLM_LARGE_MMPROJ="${LLM_LARGE_MMPROJ:-/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/mmproj-Qwen3.8-27B-BF16.gguf}"
retire_halogen_large_quadlet() {
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
@@ -1155,7 +1158,7 @@ deploy_llm_large() {
for f in "$LLM_LARGE_MODEL" "$LLM_LARGE_DRAFT" "$LLM_LARGE_MMPROJ"; do
if [[ ! -f "$f" ]]; then
echo "ERROR: payload absent at $f" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-kingjones.sh" >&2
echo "frank-large: fetch it: bash scripts/download-frank-large-kingjones-uncensored.sh" >&2
missing=1
fi
done
@@ -1172,7 +1175,7 @@ deploy_llm_large() {
# down with it. The pattern never appears in the deploying shell's own
# command line, so -f is safe here (it is not in `bash deploy-remote.sh`).
run_privileged systemctl stop frank-large.service 2>/dev/null || true
pkill -f 'Qwen3.8-27B-Q8_0_ROCMFPX_AGENT' 2>/dev/null || true
pkill -f 'Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT' 2>/dev/null || true
# Nothing may start until the predecessor has released the port. On
# 2026-09-16 the cutover failed exactly here: the old server logged
# "Deactivated successfully", the next engine tried to bind 7770 a second