dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
wt/20261020-213526
serverAppConfigs/serverAppConfigs-docs/LLM/Dankclaw System Specs.md
T
Frank 6215f1a6c8 docs(vault): halogen has an NPU lane on the pin we already run, and it is greedy-only 
- frank-dragoon: measured the response_format contract. json_schema and json_object answer
  schema-valid JSON at temperature 0 and draw a 400 at any sampled temperature, and because
  the sampler default is server-side an absent temperature is sampled. The page claimed
  "response_format is unsupported", which sends the next agent past a working feature.
- frank-dragoon: new "The NPU lane" section. The pinned 0.16.2 image already ships the
  five-program AIE catalog including qwen3-embedding-0.6b, so /v1/embeddings is one env var
  plus the accel device, the distro-XRT mount and the fabric-clock hold; embeddinggemma-2
  needs 0.18.3+, the stock-kernel-driver memory-budget fix 0.18.8+. Host census recorded:
  no /opt/xilinx/xrt, distro XRT 1:2.21.75 with libxrt_driver_xdna, clocks unheld.
- Open Notebook: scriptwriter probe verdict — greedy-only, and open-notebook's model record
  is {credential, name, provider, type} with no per-row config, so it cannot ask for greedy;
  frank-archon stays the writer. The embedding gap now has a named door instead of "build a
  new llama.cpp slot".
- Frank TTS: "the NPU is not a door at all" was too broad. It is not a door for the TTS
  family; halogen's precompiled catalog is a door on the same device.
- System Specs: NPU runtime row corrected to the distro XRT actually installed, and the
  stale "frank-zealot serves 27B" line retired in favour of [[frank-zealot]] as sole owner.

Tests: ./tests/run.sh, 455 passed.
2026-10-19 16:40:48 -07:00

117 lines
7.7 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
type: reference
description: "Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference."
tags:
- llm
timestamp: 2026-09-03
---
# Dankclaw System Specs
## Server Overview
Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
## Hardware Specifications
### CPU
| Property | Value |
|----------|-------|
| Model | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
| Architecture | x86_64 |
| Cores | 16 (1 socket, 1 thread per core) |
| Max MHz | 5187.50 |
| Min MHz | 625.00 |
| NUMA Nodes | 1 (node 0, 124129 MB — `Node 0 MemTotal: 127108080 kB`, read 2026-10-18) |
### GPU (Integrated)
| Property | Value |
|----------|-------|
| Model | AMD Radeon 8060S (Strix Halo) |
| Device ID | 0x1586 |
| Architecture | gfx1151 (Strix Halo) |
| GPU-Visible Memory | ~114 GiB GTT aperture (the budget; `mem_info_gtt_total` = 122,402,963,456 B read 2026-10-18; rocm-smi's smaller SVM carve-out is not the budget) |
| Physical RAM | 121 GiB visible to the OS (`free -g` total 121, read 2026-10-18) on a 128 GB part — 8 × 16 GB LPDDR5 ( §hardware). The 64 GiB this page recorded as "corrected 2026-08-17" (from 32 GB) is stale |
### NPU
| Property | Value |
|----------|-------|
| Model | AMD XDNA2 (Ryzen AI) |
| Rating | 50 TOPS |
| Device | /dev/accel/accel0 |
| Driver | amdxdna 0.7.0, in-tree — bound since 2026-10-18 (`lspci -k -s c6:00.1` → `Kernel driver in use: amdxdna`; journal 2026-10-18 01:25:14 `Initialized amdxdna_accel_driver 0.7.0 for 0000:c6:00.1 on minor 0`) |
| Firmware | 1.1.2.65 (`/usr/lib/firmware/amdnpu/17f0_11/npu_7.sbin` → `npu.sbin.1.1.2.65.zst`, loaded at boot 2026-10-18 01:25:13) |
| Runtime | Distro XRT only: `libxrt2` + `libxrt-npu2` 1:2.21.75 supply `libxrt_coreutil.so.2`, `libxrt_core.so.2` and the NPU plugin `libxrt_driver_xdna.so.2` under `/usr/lib/x86_64-linux-gnu`; **no `/opt/xilinx/xrt`** (census 2026-10-19), no Ryzen AI Python stack. FastFlowLM 0.9.45 + AMD's XRT (installed 2026-08-16) was **retired 2026-09-11** with the `amd_iommu=off` decision — see [[dankclaw-iommu-translation-off]]; the package remains at `/home/dank/GitPackages/fastflowlm_0.9.45_ubuntu26.04_amd64.deb` |
| Status | Not in service — no NPU consumer is deployed. The device itself is usable since the 2026-10-18 `iommu=pt` boot (`/dev/accel/accel0` present, `crw-rw---- root:render 261,0`); Gitea issue #3 tracks enablement. The only runtime here that ships precompiled programs this device can execute is halogen's — see [[frank-dragoon]] §The NPU lane |
This section is device inventory, not a payload record. The name `frank-zealot` served the FLM NPU endpoint until
2026-09-11 and has served Bonsai-2 on the RTX 3090 through the llama.cpp PrismML fork since; the checkpoint, the
shape and every era are on [[frank-zealot]], and any figure restated here is the next thing to go stale.
### ROCm
| Property | Value |
|----------|-------|
| ROCk Module | Loaded |
| Runtime Version | 1.18 |
| ROCm Version | 7.2.4 system (`/opt/rocm/.info/version` = 7.2.4, read 2026-10-18). The TheRock 7.15 SDK that lived in the CIRU venv left the serving path when CIRU was retired 2026-08-22 |
| Agent Name | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
| GPU Arch | gfx1151 (Strix Halo) |
| GPU Global Memory | ~114 GiB GTT aperture (the budget) |
### Memory
| Property | Value |
|----------|-------|
| System RAM | 121 GiB (shared CPU + GPU; `free -g` total 121, `MemTotal 127108080 kB`, 2026-10-18) |
| GPU-Visible (ROCm) | ~114 GiB GTT aperture (`mem_info_gtt_total` 122,402,963,456 B — the budget) |
| Swap | 64 GB `/swap.img` (`swapon`, 2026-10-18; `free` reports 63 GiB) |
| TTM page cap | `ttm.pages_limit = 29883536` (114 GiB of 4 KiB pages) and `page_pool_size=29883536`, persisted in `/etc/modprobe.d/ttm.conf`, applied live 2026-08-25 and re-read 2026-10-18 (`/sys/module/ttm/parameters/pages_limit`). The kernel default is half of RAM, so in the firmware-VGM era (~30 GiB visible) the cap was ~15 GiB and a 28 GB payload plus a 262k KV cache could not load — this is the row every other page should cite |
| /dev/shm | 61 GiB tmpfs (half of RAM, `df /dev/shm`, 2026-10-18) |
### Storage
| Filesystem | Size | Used | Avail | Use% |
|------------|------|------|-------|------|
| / (nvme0n1p2) | 1.9T | 1.4T | 424G | 77% |
### Network
| Interface | Type | Speed | IP | Notes |
|-----------|------|-------|----|-------|
| eno1 | onboard | 2500 Mb/s (2.5 GbE) | 192.168.0.143 (DHCP) | Carried the RPC path to Dankserver during the 2026-08-17 pipeline-parallelism trial; no RPC listener runs today (`ss -ltn` shows no :50052, 2026-10-18). Default route via 192.168.0.1. |
| enx00e04cc2e1e5 | USB-C to Ethernet adapter | 1000 Mb/s | 192.168.0.116 | UNPLUGGED 2026-08-17. While present, the route to 192.168.0.235 went through this adapter (lower route metric), so all RPC traffic was capped at 1 GbE. See section 0. |
| docker0 | bridge | - | 172.17.0.1 | Usually down. |
The L2 gateway/switch at 192.168.0.1 (MAC f0:4b:8a:65:c5:40) is at
least 2.5 GbE-capable. Whether it is 10G-capable is unverified — check
the device itself before buying 10G USB-C adapters; otherwise a direct
cable between claw and server is required for 10G.
### OS and Software
| Property | Value |
|----------|-------|
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
| Kernel | 7.0.0-38-generic (`uname -r`, 2026-10-18) |
| Boot line | `iommu=pt` since 2026-10-18 (`/proc/cmdline`, no `amd_iommu` token); eras before: `amd_iommu=off` 2026-09-11 → 10-18, IOMMU default-on before that — [[dankclaw-iommu-translation-off]] |
| Python | 3.14.4 (system, `python3 --version`, 2026-10-18). The 3.12.13 CIRU venv left with that runtime (retired 2026-08-22) |
| CIRU runtime | ciru-ai/ROCmFPX — **retired 2026-08-22** ([[frank-dragoon]]); not the serving runtime |
| uv | 0.12.8 (`~/.local/bin/uv --version`, 2026-10-18) |
## Runtime Relevance
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
- The GPU budget is the GTT aperture: ~114 GiB (`mem_info_gtt_total` in `/sys/class/drm/card1/device/` = 122,402,963,456 B; `card1` is the amdgpu iGPU, `PCI_ID=1002:1586`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
- The gufo-era note that the active runtime links against the host ROCm (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH belongs to that retired source build. The serving runtimes do not depend on the host ROCm that way: halogen runs inside its digest-pinned image, and `frank-zealot` is a CUDA build on the 3090. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) was never in the serving path either.
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
- 424 GB free on `/` (`df -h`, 2026-10-18) is ample for model storage.
## See Also
- [[frank-dragoon]]
- [[Shared Memory Architecture Dankclaw]]
Reference in new issue View Git Blame Copy Permalink