Files

T

Frank 4462aa992c Swap LLM fleet to Ornith CRACK (small) and kingjones ROCmFP4 STRIX MTP (large); apply Qwen-Sharp chat template

- frank-default-small (Dankserver): Ornith-1.5-35B-A3B-CRACK Q8_0, bundled
  native draft-mtp (n_max 4), MoE expert FFN tensors pinned to CPU via an
  explicit --override-tensor \.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU
  (replaces --n-cpu-moe), Qwen-Sharp chat template.
- frank-default-large (Dankclaw): kingjones Qwen3.8-27B-Uncensored-Q6_0
  ROCmFP4 AGENT dense GGUF, standalone mtp-Qwen3.8-27B-Q4_0 draft head
  (--model-draft, draft-mtp n_max 4), Qwen-Sharp chat template. Drop the
  retired PromptForge .pfs sidecar lanes.
- Track scripts/chat-template.jinja (v22.4.0 Sharp) and apply via
  --chat-template-file on both services.
- Docs: new kingjones-strix-mtp and dealign-ornith-crack runbook pages;
  update index, deployment, and both system specs; models.json names.

Smoke-tested both models on the real hardware on scratch ports before
commit: model loads, MTP draft attaches, Sharp template applies, chat
completion returns.

2026-08-24 21:35:11 -07:00

4.3 KiB

Raw Blame History

Dankclaw System Specs

Server Overview

Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool.

Hardware Specifications

CPU

Property	Value
Model	AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
Architecture	x86_64
Cores	16 (1 socket, 1 thread per core)
Max MHz	5187.50
Min MHz	625.00
NUMA Nodes	1 (node 0, 31213 MB)

GPU (Integrated)

Property	Value
Model	AMD Radeon 8060S (Strix Halo)
Device ID	0x1586
Architecture	gfx1151 (Strix Halo)
GPU-Visible Memory	64 GiB (rocm-smi; SVM-backed, see the memory model below)
Physical RAM	64 GiB (61 GiB visible; upgraded from 32 GB, corrected 2026-08-17)

NPU

Property	Value
Model	AMD XDNA2 (Ryzen AI)
Rating	50 TOPS
Device	/dev/accel/accel0
Driver	amdxdna 0.7 (in-tree, kernel 7.0.0-29-generic)
Firmware	1.1.2.65
Runtime	FastFlowLM 0.9.45 + XRT (installed 2026-08-16; retired 2026-08-17)
Status	Not in service. Research only. See FastFlowLM NPU Inference

The NPU is retired; the 52625 port and the frank-default-small name now serve the CIRU ROCmFPX llama.cpp runtime (see ciru-activefpx-promptforge).

ROCm

Property	Value
ROCk Module	Loaded
Runtime Version	1.18
ROCm Version	7.2.4 system (/opt/rocm) + TheRock 7.15 SDK in the CIRU venv (used for the runtime)
Agent Name	AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
GPU Arch	gfx1151 (Strix Halo)
GPU Global Memory	64 GiB (rocm-smi; SVM-backed)

Memory

Property	Value
System RAM	64 GiB (shared CPU + GPU; 61 GiB visible)
GPU-Visible (ROCm)	64 GiB (rocm-smi; SVM-backed)
Swap	8.0 GiB
/dev/shm	~15 GiB (half of RAM)

Storage

Filesystem	Size	Used	Avail	Use%
/ (nvme0n1p2)	1.9T	1.3T	545G	70%

Network

Interface	Type	Speed	IP	Notes
eno1	onboard	2500 Mb/s (2.5 GbE)	192.168.0.143 (DHCP)	Carries the RPC path to Dankserver since 2026-08-17. Default route via 192.168.0.1.
enx00e04cc2e1e5	USB-C to Ethernet adapter	1000 Mb/s	192.168.0.116	UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See Pipeline Parallelism/Ethernet Bandwidth Math section 0.
docker0	bridge	-	172.17.0.1	Usually down.

The L2 gateway/switch at 192.168.0.1 (MAC f0:4b:8a:65:c5:40) is at least 2.5 GbE-capable. Whether it is 10G-capable is unverified — check the device itself before buying 10G USB-C adapters; otherwise a direct cable between claw and server is required for 10G.

OS and Software

Property	Value
OS	Ubuntu 26.04 LTS (Resolute Raccoon)
Kernel	7.0.0-29-generic
Python	3.14.4 (system), 3.12.13 (CIRU venv)
CIRU runtime	ciru-ai/ROCmFPX v2.3 (llama-server, ROCm 7.15 SDK)
uv	0.12.2

CIRU Runtime Relevance

Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
The GPU budget is the rocm-smi figure: 64 GiB VRAM reported by rocm-smi (SVM-backed). The older 96 GiB figure was the addressable SVM pool, not the budget. Trust rocm-smi and the runtime; do not use free or dmidecode for the GPU budget.
The active service is frank-default-large on port 7770 (moved from 52625 on 2026-08-22): CIRU ROCmFPX llama.cpp serving kingjones/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP (Q6 AGENT dense, 262144 ctx, standalone MTP Q4_0 head). See kingjones-strix-mtp.
The runtime requires ROCm 7.15 (model card requirement). The system ROCm is 7.2.4; the retired ling3-ciru venv bundles a self-contained TheRock ROCm 7.15 SDK that the serve script puts first in LD_LIBRARY_PATH.
The retired vLLM venvs (/home/dank/vllm-venv, /home/dank/vllm-vk-venv) were moved to /home/dank/.AGENTTRASH/ on 2026-08-18.
1.1 TB free disk space is ample for model storage.

4.3 KiB Raw Blame History