0 Fork 0

Files

T

Frank 968066ff23 docs: boot policy (only PP units enabled), deploy.sh dankserver hazard, stale-claim cleanup

- deployment.md: Boot Policy (2026-08-17) section (disabled units table,
  re-enable recipes, stale frank-restart-small.sh helper, broken
  deploy.sh dankserver target); Update Dankserver -> frank-pp-worker;
  condensed retired 35B history block and 2026-08-13 incident
- AGENTS.md: boot-policy note, target map fix (Q6_K, split 0.68),
  deploy.sh dankserver warning, obsolete quirk entries condensed
- README.md: rewrite active services/models tables (PP head + worker)
- index.md: topology fix (qwythos:9b, Q6_K split 0.68, NPU disabled)
- qwen3.8-spec-matrix.md: superseded configs merged into compact
  history; stale inline env block removed; bundled-head MTP note
- FastFlowLM, PP Deployment: boot-policy + enablement notes
- System Specs x2, DFlash doc, 35B FP8 doc, Ling docs: stale active
  claims fixed, retired banners, never-shipped plan stubbed

2026-08-17 03:30:00 -07:00

3.7 KiB

Raw Blame History

Dankclaw System Specs

Server Overview

Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool.

Hardware Specifications

CPU

Property	Value
Model	AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
Architecture	x86_64
Cores	16 (1 socket, 1 thread per core)
Max MHz	5187.50
Min MHz	625.00
NUMA Nodes	1 (node 0, 31213 MB)

GPU (Integrated)

Property	Value
Model	AMD Radeon 8060S (Strix Halo)
Device ID	0x1586
Architecture	gfx1151 (Strix Halo)
GPU-Visible Memory	96 GiB (driver VRAM ceiling, overcommitted)
Physical RAM	30 GiB (shared CPU + GPU)

NPU

Property	Value
Model	AMD XDNA2 (Ryzen AI)
Rating	50 TOPS
Device	/dev/accel/accel0
Driver	amdxdna 0.7 (in-tree, kernel 7.0.0-29-generic)
Firmware	1.1.2.65
Runtime	FastFlowLM 0.9.45 + XRT (installed 2026-08-16)
Status	Not in service. Research only. See FastFlowLM NPU Inference

The NPU shares the 30 GiB unified pool. FLM allocates roughly half of system RAM for NPU use (~15 GiB here). It is idle in production.

ROCm

Property	Value
ROCk Module	Loaded
Runtime Version	1.18
ROCm Version	7.2.4 (from AMD repos)
Agent Name	AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
GPU Arch	gfx1151 (Strix Halo)
GPU Global Memory	96 GiB (driver-reported ceiling; physical is 30 GiB)

Memory

Property	Value
System RAM	30 GiB (shared CPU + GPU)
GPU-Visible (ROCm)	96 GiB (driver ceiling; ROCm pool 1 is 31962252 KB)
Swap	8.0 GiB
/dev/shm	~15 GiB (half of RAM)

Storage

Filesystem	Size	Used	Avail	Use%
/ (nvme0n1p2)	1.9T	661G	1.1T	38%

OS and Software

Property	Value
OS	Ubuntu 26.04 LTS (Resolute Raccoon)
Kernel	7.0.0-29-generic
Python	3.14.4 (system), 3.12.13 (CIRU vLLM venv)
vLLM	0.26.1rc1.dev467+g838616875.rocm715 (CIRU venv, Python 3.12)
uv	0.12.2

VLLM Relevance

Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
ROCm reports 96 GiB of GPU-visible memory (overcommitted). The current vLLM service runs at --gpu-memory-utilization 0.95, which fits the CIRU INT4 weights and 256k KV cache (about 90 GiB used of the 96 GiB frame buffer).
Without the ROCm VRAM patch, ROCm clamps APU memory to 50% (see scripts/patches/rocm_patched.py on the remote). The patch restores the real GTT ceiling minus an 8 GB safety margin.
Only 8 GiB swap. Modern vLLM uses --kv-offloading-size instead of the removed --swap-space flag.
The active service on Dankclaw is frank-default-large.service: the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. The vLLM-ROCm CIRU Ling path was planned but never shipped. See deployment.
ROCm 7.2.4 from AMD repos. The CIRU venv bundles a self-contained ROCm 7.15 toolchain; vLLM ROCm support requires ROCm 6.3+.
Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. The CIRU venv uses Python 3.12.
Radeon 8060S falls under Ryzen AI MAX series (gfx1151). vLLM lists this as supported.
1.1 TB free disk space is ample for model storage.
libmpi_cxx.so.40 is missing from the system. Install openmpi-bin or provide a workaround for torch MPI dependency.

3.7 KiB Raw Blame History