chore(llm): add dankstatus.sh — one verified parse path for Dankserver state

The ad-hoc status commands kept producing wrong readings, most seriously
reporting "MemAvailable 0.0 GiB" on a host that actually had 240 GiB free.
Cause: /proc/meminfo separates key from value with a TAB, so

  grep MemAvailable /proc/meminfo | tr -s " " | cut -d" " -f2

returns an empty field and awk prints 0.0. A false "host is out of memory"
reading is worse than no reading, because it invites an unnecessary abort.

dankstatus.sh parses on a colon split (tab-agnostic), cross-checks
MemAvailable against MemTotal before printing either, and refuses to print
a zero value. It also pins the rocm-smi CSV column order (device, total,
used) which was previously read swapped, and takes only the first line of
systemctl is-active/is-enabled, which emit more than one line here.

Incidental finding while validating: rocm-smi reports card2 (the 7700X iGPU)
as 0.50 GiB total, not the 123.0 GiB torch.cuda reports. torch was exposing
the GTT aperture. The device is smaller than documented, not larger.

Tests: 346/346 pass (no test surface touched).
This commit is contained in:
Frank committed 2026-09-28 23:27:50 -07:00
1 parent b4b6777e49
commit 32abc8d543
8 files changed
+525 -146

No files matched your search

+46 -23
View File
@@ -1,36 +1,52 @@
#!/usr/bin/env bash
# Serve unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL as frank-large on Dankclaw's
# Strix Halo iGPU (gfx1151), using gufo (gufo-org/gufo) — the inference
# engine built specifically for this hardware.
# Serve Swift-1.5-Qwen3.8-27B-Uncensored (abliterated) as frank-large on
# Dankclaw's Strix Halo iGPU (gfx1151), using gufo (gufo-org/gufo) — the
# inference engine built specifically for this hardware.
#
# Runtime: /home/dank/GitPackages/gufo-v0.1.1/bin/gufo, built from source
# against the host ROCm by scripts/build-gufo.sh. It replaces the stock
# llama.cpp prebuilt (unsloth b11030) that served this endpoint from
# 2026-09-21; payload, port and alias are unchanged, so no client moved.
# against the host ROCm by scripts/build-gufo.sh. It replaced the stock
# llama.cpp prebuilt (unsloth b11030) on 2026-09-28; port, alias and client
# contract did not move.
#
# Payload: ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF's
# UD-Q8_K_XL, fetched by scripts/download-frank-large-swift.sh. It replaced
# unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL on 2026-09-28. The two are structurally
# interchangeable for this engine — same `qwen35` architecture, the same 866
# tensors in the same order, the same per-tensor Q8_0/F32/BF16 layout, the
# same MTP head at blk.64 — so the swap needed no engine change. The fine-tune
# is abliterated (uncensored) and carries the SWIFT OPEN LICENSE 1.0, which
# is not an OSI licence.
#
# Shape: one session, full 262144 context (the model's native window). The
# 27B dense weights (29.30 GiB) + the DFlash2 drafter (1.07 GiB) + KV +
# compute buffers fit the iGPU's GTT aperture (~114 GiB); gufo's own
# Q8/AR table measures 50.58 GiB peak at this context capacity.
# 27B dense weights (29.3 GiB) + the DFlash2 drafter (1.07 GiB) + KV +
# compute buffers fit the iGPU's GTT aperture (~114 GiB); a load at this
# context measured 66.5 GiB of the 116.7 GiB device.
#
# Spec method: DFlash2 (a block-diffusion drafter), gufo's adaptive block
# policy, capped at DRAFT_TOKENS proposals per step. The drafter is the
# z-lab Qwen3.8-27B-DFlash2-Q4_K_M and it has a sliding window of 2048
# (small draft KV).
# (small draft KV). It is UNCHANGED by the model swap and still lives beside
# the previous target, which is why DRAFTER_DIR is its own tunable.
#
# Sampling: greedy (--temperature 0). That is what this endpoint has always
# served, and it keeps the swap to a pure runtime change; gufo's per-model
# recommendation for Qwen3.8 thinking is sampled (1.0 / 0.95 / top-k 20) and
# is a separate decision.
# Sampling: greedy (--temperature 0) server-side, as every frank-large has
# served. The harness overrides it per request via extraBody (1.0 / 0.95 /
# top-k 20).
#
# Template: gufo compiles the official Qwen Jinja into the binary and takes
# no --chat-template-file. Its Qwen27B default is thinking ON at xhigh effort
# with prior reasoning preserved, which is the contract the fleet already
# depends on, so --think auto + --preserve-thinking on is spelled out here
# rather than left implicit.
# NO --chat-template-file — and it is not a Jinja engine at all
# (src/models/qwen/chat_template.cpp is a hand-written C++ renderer), so the
# fleet's scripts/qwen-sharp-v22.4.0.jinja cannot be supplied to it. That law
# was given up when this endpoint moved to gufo on 2026-09-28, by decision:
# the terse system-prompt injection, the in-band <|think_*|> control, the
# json|xml tool_call_format switch and the tool-argument truncation are all
# absent. gufo's own renderer keeps the effort instruction strings and
# preserve_thinking, which are the parts the harness depends on. Accepted
# 2026-09-28; recovering the terse prompt from the client side is the
# cheapest partial restore.
#
# Text-only: gufo auto-discovers a mmproj beside the model and this payload
# has none, so the endpoint stays text-only.
# has none (the base repo ships one separately, unused), so the endpoint
# stays text-only.
#
# Tunables come from frank-large.env unless FRANK_LARGE_ENV_FILE overrides
# it; anything not set there falls through to the defaults below.
@@ -46,7 +62,7 @@ fi
# --- defaults (used when the env file does not set them) ------------------
: "${RUNTIME_DIR:=/home/dank/GitPackages/gufo-v0.1.1}"
: "${MODEL_DIR:=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL}"
: "${MODEL_DIR:=/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL}"
: "${ALIAS:=frank-large}"
: "${HOST:=0.0.0.0}"
: "${PORT:=7770}"
@@ -56,8 +72,15 @@ fi
: "${DRAFT_POLICY:=adaptive}"
GUFO_BIN="$RUNTIME_DIR/bin/gufo"
MODEL_FILE="$MODEL_DIR/Qwen3.8-27B-UD-Q8_K_XL.gguf"
DRAFTER_FILE="$MODEL_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf"
# The target GGUF and the drafter are named explicitly because they do not
# share a directory any more: the target is a different repository's artifact
# and keeps its own long filename, while the DFlash2 drafter is unchanged and
# still lives beside the previous target. MODEL_FILE/DRAFTER_FILE default to
# the deployed payload, so the env file only has to state them when it moves.
: "${MODEL_FILE:=$MODEL_DIR/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf}"
: "${DRAFTER_DIR:=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL}"
: "${DRAFTER_FILE:=$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf}"
if [[ ! -x "$GUFO_BIN" ]]; then
echo "runtime not found at $GUFO_BIN" >&2
@@ -66,7 +89,7 @@ if [[ ! -x "$GUFO_BIN" ]]; then
fi
for f in "$MODEL_FILE" "$DRAFTER_FILE"; do
if [[ ! -f "$f" ]]; then
echo "missing asset $f (run scripts/download-frank-large.sh)" >&2
echo "missing asset $f (run the matching scripts/download-frank-large*.sh)" >&2
exit 1
fi
done