Frank
32abc8d543
The ad-hoc status commands kept producing wrong readings, most seriously reporting "MemAvailable 0.0 GiB" on a host that actually had 240 GiB free. Cause: /proc/meminfo separates key from value with a TAB, so grep MemAvailable /proc/meminfo | tr -s " " | cut -d" " -f2 returns an empty field and awk prints 0.0. A false "host is out of memory" reading is worse than no reading, because it invites an unnecessary abort. dankstatus.sh parses on a colon split (tab-agnostic), cross-checks MemAvailable against MemTotal before printing either, and refuses to print a zero value. It also pins the rocm-smi CSV column order (device, total, used) which was previously read swapped, and takes only the first line of systemctl is-active/is-enabled, which emit more than one line here. Incidental finding while validating: rocm-smi reports card2 (the 7700X iGPU) as 0.50 GiB total, not the 123.0 GiB torch.cuda reports. torch was exposing the GTT aperture. The device is smaller than documented, not larger. Tests: 346/346 pass (no test surface touched).
46 lines
2.2 KiB
Bash
46 lines
2.2 KiB
Bash
# frank-large — gufo (gufo-org/gufo) serving
|
|
# Swift-1.5-Qwen3.8-27B-Uncensored (abliterated) on the Strix Halo iGPU.
|
|
#
|
|
# These are the tunables read by scripts/serve-frank-large.sh. The flag
|
|
# list is owned by the serve script; this file only sets fleet values.
|
|
#
|
|
# DO NOT name a new tunable TEMP, HOST or GPU_MAX_HEAP_SIZE: the HIP
|
|
# runtime reads those bare (the 2026-08-13 TEMP incident). HOST is the one
|
|
# exception because the serve script itself consumes it for --host.
|
|
|
|
# The target payload (scripts/download-frank-large-swift.sh): the serve
|
|
# script refuses to start without the main GGUF and the DFlash2 drafter.
|
|
# Swapped in from unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL on 2026-09-28. The two
|
|
# are structurally interchangeable for gufo (same qwen35 architecture, the
|
|
# same 866 tensors in the same order, the same per-tensor Q8_0/F32/BF16
|
|
# layout, the same MTP head), so only this path changed.
|
|
MODEL_DIR=/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL
|
|
MODEL_FILE=/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
|
|
|
|
# The DFlash2 drafter is UNCHANGED by the model swap and still lives beside
|
|
# the previous target, so it carries its own directory. It has a sliding
|
|
# window of 2048, which is what keeps its KV small (~40 MiB).
|
|
DRAFTER_DIR=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL
|
|
DRAFTER_FILE=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf
|
|
|
|
# The gufo runtime, built from source by scripts/build-gufo.sh.
|
|
RUNTIME_DIR=/home/dank/GitPackages/gufo-v0.1.1
|
|
|
|
# Fleet endpoint: the same port and alias every frank-large has served, so
|
|
# no client, probe or models.yml entry changes address.
|
|
ALIAS=frank-large
|
|
HOST=0.0.0.0
|
|
PORT=7770
|
|
|
|
# One session, full 262144 context (the model's native window).
|
|
CONTEXT_SIZE=262144
|
|
SESSIONS=1
|
|
|
|
# DFlash2 spec decode (a block-diffusion drafter). gufo sizes the draft
|
|
# block itself; DRAFT_TOKENS caps the proposals per step (gufo's default is
|
|
# 7, chosen over llama.cpp's fixed depth 4 because gufo's adaptive policy
|
|
# shortens the block when acceptance drops) and DRAFT_POLICY is the block
|
|
# length controller.
|
|
DRAFT_TOKENS=7
|
|
DRAFT_POLICY=adaptive
|