docs: fact-check knowledge base against live machines and vLLM source
Correct stale and incorrect facts after exhaustive verification: - Dankclaw: NUMA node 0 = 31213 MB; disk 661G/1.1T/38%; vLLM 0.26.0 in /home/dank/vllm-rocm-venv (not vllm-env); clarify 96 GiB GPU-visible is a driver overcommit ceiling over 30 GiB physical (confirmed via sysfs mem_info_vram_total=103079215104) - Dankserver: disk 729G/140G free/84%; /mnt/entropy 910G free; active service is vLLM EngineCore (22 GB), not llama-server; CPU model Ryzen 7 7700X - index.md: Dankclaw 30GiB phys / 96GiB GPU-visible (was '61 GB unified'); add deployment topology mermaid diagram - vLLM defaults (from source): gpu_memory_utilization=0.92, max_num_seqs=128, swap_space removed (now --kv-offloading-size) across Overview/Index/RAM/Hybrid - Quantization: FP8 not native on Ampere (RTX 3090) or RDNA 3.5 (gfx1151); add per-model hardware quantization notes; AWQ/GPTQ limited on ROCm - Speculative decoding: add DFlash, DSpark, MTP, EAGLE3 to feature list - Confirm Qwen3.6-27B and Qwen3.6-35B-A3B specs are correct (HF-verified) - Production note: Dankclaw primary llama.cpp service runs ROCmFPX Ling-3.0-flash RocmFP4 MTP, not vLLM Qwen3.6-27B
This commit is contained in:
1 parent
841d872aa7
commit
3a64c8f3dd
10 files changed
+86
-49
No files matched your search
@@ -14,7 +14,7 @@ Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 wit
|
||||
| Cores | 16 (1 socket, 1 thread per core) |
|
||||
| Max MHz | 5187.50 |
|
||||
| Min MHz | 625.00 |
|
||||
| NUMA Nodes | 1 (node 0, 62909 MB) |
|
||||
| NUMA Nodes | 1 (node 0, 31213 MB) |
|
||||
|
||||
### GPU (Integrated)
|
||||
|
||||
@@ -23,7 +23,7 @@ Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 wit
|
||||
| Model | AMD Radeon 8060S (Strix Halo) |
|
||||
| Device ID | 0x1586 |
|
||||
| Architecture | gfx1151 (Strix Halo) |
|
||||
| GPU-Visible Memory | 96 GiB (ROCm addressable range) |
|
||||
| GPU-Visible Memory | 96 GiB (driver VRAM ceiling, overcommitted) |
|
||||
| Physical RAM | 30 GiB (shared CPU + GPU) |
|
||||
|
||||
### ROCm
|
||||
@@ -35,14 +35,14 @@ Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 wit
|
||||
| ROCm Version | 7.2.4 (from AMD repos) |
|
||||
| Agent Name | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
|
||||
| GPU Arch | gfx1151 (Strix Halo) |
|
||||
| GPU Global Memory | 96 GiB (overcommitted, physical is 30 GiB) |
|
||||
| GPU Global Memory | 96 GiB (driver-reported ceiling; physical is 30 GiB) |
|
||||
|
||||
### Memory
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| System RAM | 30 GiB (shared CPU + GPU) |
|
||||
| GPU-Visible (ROCm) | 96 GiB (overcommitted addressable range) |
|
||||
| GPU-Visible (ROCm) | 96 GiB (driver ceiling; ROCm pool 1 is 31962252 KB) |
|
||||
| Swap | 8.0 GiB |
|
||||
| /dev/shm | ~15 GiB (half of RAM) |
|
||||
|
||||
@@ -50,7 +50,7 @@ Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 wit
|
||||
|
||||
| Filesystem | Size | Used | Avail | Use% |
|
||||
|------------|------|------|-------|------|
|
||||
| / (nvme0n1p2) | 1.9T | 391G | 1.4T | 22% |
|
||||
| / (nvme0n1p2) | 1.9T | 661G | 1.1T | 38% |
|
||||
|
||||
### OS and Software
|
||||
|
||||
@@ -59,18 +59,20 @@ Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 wit
|
||||
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
|
||||
| Kernel | 7.0.0-29-generic |
|
||||
| Python | 3.14.4 (system), 3.12.13 (vLLM venv) |
|
||||
| vLLM | 0.26.0+rocm723 |
|
||||
| vLLM | 0.26.0 (in `/home/dank/vllm-rocm-venv`, Python 3.12) |
|
||||
| uv | 0.12.2 |
|
||||
|
||||
## VLLM Relevance
|
||||
|
||||
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
|
||||
- ROCm reports 96 GiB GPU memory (overcommitted). Physical RAM is 30 GiB. vLLM sees 96 GiB and will try to use it. Set `--gpu-memory-utilization` conservatively (0.80-0.85) to avoid OOM when the OS needs memory.
|
||||
- Only 8 GiB swap. Set `--swap-space` conservatively (4-8 GiB).
|
||||
- ROCm reports 96 GiB of GPU-visible memory (overcommitted). Physical RAM is 30 GiB. The driver reports a 96 GiB VRAM ceiling (`sysfs mem_info_vram_total`), but the machine has only 30.5 GB of physical RAM. vLLM sees the large number and will try to use it. Set `--gpu-memory-utilization` conservatively (0.80-0.85) to avoid OOM when the OS needs memory.
|
||||
- Without the ROCm VRAM patch, ROCm clamps APU memory to 50% (see `scripts/patches/rocm_patched.py` on the remote). The patch restores the real GTT ceiling minus an 8 GB safety margin.
|
||||
- Only 8 GiB swap. Modern vLLM uses `--kv-offloading-size` instead of the removed `--swap-space` flag.
|
||||
- The active llama.cpp service on Dankclaw runs on the ROCmFPX runtime (`/home/dank/GitPackages/ROCmFPX/build-strix-rocmfp4/bin/llama-server`) with the Ling-3.0-flash ROCmFP4 MTP model, not regular llama.cpp.
|
||||
- ROCm 7.2.4 from AMD repos. vLLM ROCm support requires ROCm 6.3+.
|
||||
- Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. Use a Python 3.12 venv.
|
||||
- Radeon 8060S falls under Ryzen AI MAX series (gfx1151). vLLM lists this as supported.
|
||||
- 1.4 TB free disk space is ample for model storage.
|
||||
- 1.1 TB free disk space is ample for model storage.
|
||||
- `libmpi_cxx.so.40` is missing from the system. Install `openmpi-bin` or provide a workaround for torch MPI dependency.
|
||||
|
||||
## See Also
|
||||
|
||||
Reference in new issue
Block a user