docs: reflect CIRU runtime on Dankclaw in specs and deployment pages
- Dankclaw System Specs: correct RAM to 64 GiB (61 visible), GPU budget to the rocm-smi 64 GiB SVM figure, and replace the vLLM relevance section with the CIRU ROCmFPX runtime facts. Note the retired vLLM venvs in .AGENTTRASH. Refresh the disk figures. - deployment.md: update the dankclaw target row (CIRU runtime).
This commit is contained in:
Frank
@@ -2,7 +2,7 @@
|
||||
|
||||
## Server Overview
|
||||
|
||||
Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool.
|
||||
Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool.
|
||||
## Hardware Specifications
|
||||
|
||||
### CPU
|
||||
@@ -23,8 +23,8 @@ Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 wit
|
||||
| Model | AMD Radeon 8060S (Strix Halo) |
|
||||
| Device ID | 0x1586 |
|
||||
| Architecture | gfx1151 (Strix Halo) |
|
||||
| GPU-Visible Memory | 96 GiB (driver VRAM ceiling, overcommitted) |
|
||||
| Physical RAM | 30 GiB (shared CPU + GPU) |
|
||||
| GPU-Visible Memory | 64 GiB (rocm-smi; SVM-backed, see the memory model below) |
|
||||
| Physical RAM | 64 GiB (61 GiB visible; upgraded from 32 GB, corrected 2026-08-17) |
|
||||
|
||||
### NPU
|
||||
|
||||
@@ -38,7 +38,7 @@ Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 wit
|
||||
| Runtime | FastFlowLM 0.9.45 + XRT (installed 2026-08-16; **retired 2026-08-17**) |
|
||||
| Status | Not in service. Research only. See [[FastFlowLM NPU Inference]] |
|
||||
|
||||
The NPU shares the 30 GiB unified pool. FLM allocates roughly half of system RAM for NPU use (~15 GiB here). It is idle in production. The NPU is retired; the 52625 port and the `frank-default-small` name now serve vLLM (see [[frank-redo-2026-08-17]]).
|
||||
The NPU is retired; the 52625 port and the `frank-default-small` name now serve the CIRU ROCmFPX llama.cpp runtime (see [[ciru-activefpx-promptforge]]).
|
||||
|
||||
### ROCm
|
||||
|
||||
@@ -46,17 +46,17 @@ The NPU shares the 30 GiB unified pool. FLM allocates roughly half of system RAM
|
||||
|----------|-------|
|
||||
| ROCk Module | Loaded |
|
||||
| Runtime Version | 1.18 |
|
||||
| ROCm Version | 7.2.4 (from AMD repos) |
|
||||
| ROCm Version | 7.2.4 system (/opt/rocm) + TheRock 7.15 SDK in the CIRU venv (used for the runtime) |
|
||||
| Agent Name | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
|
||||
| GPU Arch | gfx1151 (Strix Halo) |
|
||||
| GPU Global Memory | 96 GiB (driver-reported ceiling; physical is 30 GiB) |
|
||||
| GPU Global Memory | 64 GiB (rocm-smi; SVM-backed) |
|
||||
|
||||
### Memory
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| System RAM | 30 GiB (shared CPU + GPU) |
|
||||
| GPU-Visible (ROCm) | 96 GiB (driver ceiling; ROCm pool 1 is 31962252 KB) |
|
||||
| System RAM | 64 GiB (shared CPU + GPU; 61 GiB visible) |
|
||||
| GPU-Visible (ROCm) | 64 GiB (rocm-smi; SVM-backed) |
|
||||
| Swap | 8.0 GiB |
|
||||
| /dev/shm | ~15 GiB (half of RAM) |
|
||||
|
||||
@@ -64,7 +64,7 @@ The NPU shares the 30 GiB unified pool. FLM allocates roughly half of system RAM
|
||||
|
||||
| Filesystem | Size | Used | Avail | Use% |
|
||||
|------------|------|------|-------|------|
|
||||
| / (nvme0n1p2) | 1.9T | 661G | 1.1T | 38% |
|
||||
| / (nvme0n1p2) | 1.9T | 1.3T | 545G | 70% |
|
||||
|
||||
### Network
|
||||
|
||||
@@ -85,27 +85,22 @@ cable between claw and server is required for 10G.
|
||||
|----------|-------|
|
||||
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
|
||||
| Kernel | 7.0.0-29-generic |
|
||||
| Python | 3.14.4 (system), 3.12.13 (CIRU vLLM venv) |
|
||||
| vLLM | 0.26.1rc1.dev467+g838616875.rocm715 (CIRU venv, Python 3.12) |
|
||||
| Python | 3.14.4 (system), 3.12.13 (CIRU venv) |
|
||||
| CIRU runtime | ciru-ai/ROCmFPX v2.3 (llama-server, ROCm 7.15 SDK) |
|
||||
| uv | 0.12.2 |
|
||||
|
||||
## VLLM Relevance
|
||||
## CIRU Runtime Relevance
|
||||
|
||||
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
|
||||
- ROCm reports 96 GiB of GPU-visible memory (overcommitted). The current vLLM service runs at `--gpu-memory-utilization 0.95`, which fits the CIRU INT4 weights and 256k KV cache (about 90 GiB used of the 96 GiB frame buffer).
|
||||
- Without the ROCm VRAM patch, ROCm clamps APU memory to 50% (see `scripts/patches/rocm_patched.py` on the remote). The patch restores the real GTT ceiling minus an 8 GB safety margin.
|
||||
- Only 8 GiB swap. Modern vLLM uses `--kv-offloading-size` instead of the removed `--swap-space` flag.
|
||||
- The active service on Dankclaw is `frank-default-large.service`: the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. The vLLM-ROCm CIRU Ling path was planned but never shipped. See [[deployment]].
|
||||
- ROCm 7.2.4 from AMD repos. The CIRU venv bundles a self-contained ROCm 7.15 toolchain; vLLM ROCm support requires ROCm 6.3+.
|
||||
- Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. The CIRU venv uses Python 3.12.
|
||||
- Radeon 8060S falls under Ryzen AI MAX series (gfx1151). vLLM lists this as supported.
|
||||
- The GPU budget is the rocm-smi figure: 64 GiB VRAM reported by rocm-smi (SVM-backed). The older 96 GiB figure was the addressable SVM pool, not the budget. Trust rocm-smi and the runtime; do not use `free` or `dmidecode` for the GPU budget.
|
||||
- The active service is `frank-default-small` on port 52625: CIRU ROCmFPX llama.cpp serving jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (64K ctx, native MTP, PromptForge DualView). See [[ciru-activefpx-promptforge]].
|
||||
- The runtime requires ROCm 7.15 (model card requirement). The system ROCm is 7.2.4; the retired ling3-ciru venv bundles a self-contained TheRock ROCm 7.15 SDK that the serve script puts first in `LD_LIBRARY_PATH`.
|
||||
- The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
|
||||
- 1.1 TB free disk space is ample for model storage.
|
||||
- `libmpi_cxx.so.40` is missing from the system. Install `openmpi-bin` or provide a workaround for torch MPI dependency.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[VLLM Overview]]
|
||||
- [[ciru-activefpx-promptforge]]
|
||||
- [[Shared Memory Architecture Dankclaw]]
|
||||
- [[RAM CPU Offloading in VLLM]]
|
||||
- [[Running Qwen3.6 27B FP8 on Dankclaw]]
|
||||
- [[frank-redo-2026-08-17]]
|
||||
- [[FastFlowLM NPU Inference]]
|
||||
|
||||
+1
-1
@@ -9,7 +9,7 @@ Two servers run LLM inference services. Each server pulls its scripts from this
|
||||
| Target | Host | Runtime | Service | Port |
|
||||
|--------|------|---------|---------|------|
|
||||
| Dankserver | dank@dankserver | llama.cpp FastMTP (Qwen3.8-27B IQ4_XS + FastMTP 32K draft + ngram-mod) | frank-default-large | 7770 |
|
||||
| Dankclaw | dank@dankclaw | vLLM ROCm (nameistoken Qwen3.6-35B-A3B Quark W8A8 INT8, ngram spec + tool calling) | frank-default-small | 52625 |
|
||||
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v2, native MTP + PromptForge DualView) | frank-default-small | 52625 |
|
||||
|
||||
## Workflow
|
||||
|
||||
|
||||
Reference in New Issue
Block a user