Frank
136ee080c9
- Ethernet Bandwidth Math: new section 0 with live measurements (iperf 281-294 MB/s, ping RTT both paths, prefill 457.9 t/s at 2.5GbE, per-ubatch wire 20.5 MB, 98k KV snapshot 3.45 GB blocking the slot 13.6 s, validated link-speed model: 386/458/505 t/s) - System specs: NIC tables for both boxes; the 1 GbE USB-C adapter on the claw was silently routing the RPC path (unplugged 2026-08-17) - PP Deployment: fix hidden-state size (10 KiB/direction, H=5120) and RTT-vs-bandwidth claim (decode RTT-bound; prefill+snapshot BW-bound)
112 lines
4.6 KiB
Markdown
112 lines
4.6 KiB
Markdown
# Dankclaw System Specs
|
|
|
|
## Server Overview
|
|
|
|
Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool.
|
|
## Hardware Specifications
|
|
|
|
### CPU
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Model | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
|
|
| Architecture | x86_64 |
|
|
| Cores | 16 (1 socket, 1 thread per core) |
|
|
| Max MHz | 5187.50 |
|
|
| Min MHz | 625.00 |
|
|
| NUMA Nodes | 1 (node 0, 31213 MB) |
|
|
|
|
### GPU (Integrated)
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Model | AMD Radeon 8060S (Strix Halo) |
|
|
| Device ID | 0x1586 |
|
|
| Architecture | gfx1151 (Strix Halo) |
|
|
| GPU-Visible Memory | 96 GiB (driver VRAM ceiling, overcommitted) |
|
|
| Physical RAM | 30 GiB (shared CPU + GPU) |
|
|
|
|
### NPU
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Model | AMD XDNA2 (Ryzen AI) |
|
|
| Rating | 50 TOPS |
|
|
| Device | /dev/accel/accel0 |
|
|
| Driver | amdxdna 0.7 (in-tree, kernel 7.0.0-29-generic) |
|
|
| Firmware | 1.1.2.65 |
|
|
| Runtime | FastFlowLM 0.9.45 + XRT (installed 2026-08-16) |
|
|
| Status | Not in service. Research only. See [[FastFlowLM NPU Inference]] |
|
|
|
|
The NPU shares the 30 GiB unified pool. FLM allocates roughly half of system RAM for NPU use (~15 GiB here). It is idle in production.
|
|
|
|
### ROCm
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| ROCk Module | Loaded |
|
|
| Runtime Version | 1.18 |
|
|
| ROCm Version | 7.2.4 (from AMD repos) |
|
|
| Agent Name | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
|
|
| GPU Arch | gfx1151 (Strix Halo) |
|
|
| GPU Global Memory | 96 GiB (driver-reported ceiling; physical is 30 GiB) |
|
|
|
|
### Memory
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| System RAM | 30 GiB (shared CPU + GPU) |
|
|
| GPU-Visible (ROCm) | 96 GiB (driver ceiling; ROCm pool 1 is 31962252 KB) |
|
|
| Swap | 8.0 GiB |
|
|
| /dev/shm | ~15 GiB (half of RAM) |
|
|
|
|
### Storage
|
|
|
|
| Filesystem | Size | Used | Avail | Use% |
|
|
|------------|------|------|-------|------|
|
|
| / (nvme0n1p2) | 1.9T | 661G | 1.1T | 38% |
|
|
|
|
### Network
|
|
|
|
| Interface | Type | Speed | IP | Notes |
|
|
|-----------|------|-------|----|-------|
|
|
| eno1 | onboard | 2500 Mb/s (2.5 GbE) | 192.168.0.143 (DHCP) | Carries the RPC path to Dankserver since 2026-08-17. Default route via 192.168.0.1. |
|
|
| enx00e04cc2e1e5 | USB-C to Ethernet adapter | 1000 Mb/s | 192.168.0.116 | UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See [[Pipeline Parallelism/Ethernet Bandwidth Math]] section 0. |
|
|
| docker0 | bridge | - | 172.17.0.1 | Usually down. |
|
|
|
|
The L2 gateway/switch at 192.168.0.1 (MAC f0:4b:8a:65:c5:40) is at
|
|
least 2.5 GbE-capable. Whether it is 10G-capable is unverified — check
|
|
the device itself before buying 10G USB-C adapters; otherwise a direct
|
|
cable between claw and server is required for 10G.
|
|
|
|
### OS and Software
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
|
|
| Kernel | 7.0.0-29-generic |
|
|
| Python | 3.14.4 (system), 3.12.13 (CIRU vLLM venv) |
|
|
| vLLM | 0.26.1rc1.dev467+g838616875.rocm715 (CIRU venv, Python 3.12) |
|
|
| uv | 0.12.2 |
|
|
|
|
## VLLM Relevance
|
|
|
|
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
|
|
- ROCm reports 96 GiB of GPU-visible memory (overcommitted). The current vLLM service runs at `--gpu-memory-utilization 0.95`, which fits the CIRU INT4 weights and 256k KV cache (about 90 GiB used of the 96 GiB frame buffer).
|
|
- Without the ROCm VRAM patch, ROCm clamps APU memory to 50% (see `scripts/patches/rocm_patched.py` on the remote). The patch restores the real GTT ceiling minus an 8 GB safety margin.
|
|
- Only 8 GiB swap. Modern vLLM uses `--kv-offloading-size` instead of the removed `--swap-space` flag.
|
|
- The active service on Dankclaw is `frank-default-large.service`: the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. The vLLM-ROCm CIRU Ling path was planned but never shipped. See [[deployment]].
|
|
- ROCm 7.2.4 from AMD repos. The CIRU venv bundles a self-contained ROCm 7.15 toolchain; vLLM ROCm support requires ROCm 6.3+.
|
|
- Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. The CIRU venv uses Python 3.12.
|
|
- Radeon 8060S falls under Ryzen AI MAX series (gfx1151). vLLM lists this as supported.
|
|
- 1.1 TB free disk space is ample for model storage.
|
|
- `libmpi_cxx.so.40` is missing from the system. Install `openmpi-bin` or provide a workaround for torch MPI dependency.
|
|
|
|
## See Also
|
|
|
|
- [[VLLM Overview]]
|
|
- [[Shared Memory Architecture Dankclaw]]
|
|
- [[RAM CPU Offloading in VLLM]]
|
|
- [[Running Qwen3.6 27B FP8 on Dankclaw]]
|
|
- [[FastFlowLM NPU Inference]]
|