feat: initialize vLLM scripts knowledge base
- VLLM Overview: installation, CLI, API, architecture, quantization, parallelism - VLLM Index: quick reference with commands, endpoints, parameters - Dankserver System Specs: RTX 3090 24GB, 62GB RAM, Ubuntu 24.04 - Dankclaw System Specs: AMD Strix Halo APU, 61GB unified, ROCm 7.1 - RAM/CPU Offloading in VLLM: KV cache swap, gpu_memory_utilization, preemption - Hybrid CPU/GPU Processing (Dankserver): NVIDIA GPU + system RAM configuration - Shared Memory Architecture (Dankclaw): AMD unified memory and ROCm setup - Running Qwen3.6 35B A3B FP8 on Dankserver: MoE model deployment guide - Running Qwen3.6 27B FP8 on Dankclaw: dense model deployment guide
This commit is contained in:
commit
39db415348
11 files changed
+1577
No files matched your search
@@ -0,0 +1,75 @@
|
||||
# Dankclaw System Specs
|
||||
|
||||
## Server Overview
|
||||
|
||||
Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same 61 GB RAM pool.
|
||||
|
||||
## Hardware Specifications
|
||||
|
||||
### CPU
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Model | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
|
||||
| Architecture | x86_64 |
|
||||
| Cores | 16 (1 socket, 1 thread per core) |
|
||||
| Max MHz | 5187.50 |
|
||||
| Min MHz | 625.00 |
|
||||
| NUMA Nodes | 1 (node 0, 62909 MB) |
|
||||
|
||||
### GPU (Integrated)
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Model | AMD Radeon 8060S (Strix Halo) |
|
||||
| PCIe ID | c4:00.0 |
|
||||
| Architecture | Strix/Strix Halo |
|
||||
| Memory | Unified with system RAM (no separate VRAM) |
|
||||
|
||||
### ROCm
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| ROCk Module | Loaded |
|
||||
| Runtime Version | 1.1 |
|
||||
| HIP Version | 7.1.52801 |
|
||||
| Agent Name | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
|
||||
|
||||
### Memory
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| System RAM | 61 GiB (shared CPU + GPU) |
|
||||
| Swap | 8.0 GiB |
|
||||
| /dev/shm | 31 GiB |
|
||||
|
||||
### Storage
|
||||
|
||||
| Filesystem | Size | Used | Avail | Use% |
|
||||
|------------|------|------|-------|------|
|
||||
| / (nvme0n1p2) | 1.9T | 333G | 1.5T | 19% |
|
||||
|
||||
### OS and Software
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
|
||||
| Kernel | 7.0.0-28-generic |
|
||||
| Python | 3.14.4 |
|
||||
|
||||
## VLLM Relevance
|
||||
|
||||
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
|
||||
- 61 GB shared memory allows large models without VRAM constraints.
|
||||
- Only 8 GB swap. Set `--swap-space` conservatively (4-8 GB).
|
||||
- ROCm and HIP 7.1 are installed. vLLM ROCm support requires ROCm 6.3+.
|
||||
- Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. Verify compatibility.
|
||||
- Radeon 8060S falls under Ryzen AI MAX series (gfx1151/1150). vLLM lists this as supported.
|
||||
- 1.5 TB free disk space is ample for model storage.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[VLLM Overview]]
|
||||
- [[Shared Memory Architecture Dankclaw]]
|
||||
- [[RAM CPU Offloading in VLLM]]
|
||||
- [[Running Qwen3.6 27B FP8 on Dankclaw]]
|
||||
Reference in new issue
Block a user