feat: initialize vLLM scripts knowledge base

- VLLM Overview: installation, CLI, API, architecture, quantization, parallelism
- VLLM Index: quick reference with commands, endpoints, parameters
- Dankserver System Specs: RTX 3090 24GB, 62GB RAM, Ubuntu 24.04
- Dankclaw System Specs: AMD Strix Halo APU, 61GB unified, ROCm 7.1
- RAM/CPU Offloading in VLLM: KV cache swap, gpu_memory_utilization, preemption
- Hybrid CPU/GPU Processing (Dankserver): NVIDIA GPU + system RAM configuration
- Shared Memory Architecture (Dankclaw): AMD unified memory and ROCm setup
- Running Qwen3.6 35B A3B FP8 on Dankserver: MoE model deployment guide
- Running Qwen3.6 27B FP8 on Dankclaw: dense model deployment guide
This commit is contained in:
Frank committed 2026-08-05 23:49:02 -07:00
commit 39db415348
11 files changed
+1577

No files matched your search

+75
View File
@@ -0,0 +1,75 @@
# Dankclaw System Specs
## Server Overview
Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same 61 GB RAM pool.
## Hardware Specifications
### CPU
| Property | Value |
|----------|-------|
| Model | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
| Architecture | x86_64 |
| Cores | 16 (1 socket, 1 thread per core) |
| Max MHz | 5187.50 |
| Min MHz | 625.00 |
| NUMA Nodes | 1 (node 0, 62909 MB) |
### GPU (Integrated)
| Property | Value |
|----------|-------|
| Model | AMD Radeon 8060S (Strix Halo) |
| PCIe ID | c4:00.0 |
| Architecture | Strix/Strix Halo |
| Memory | Unified with system RAM (no separate VRAM) |
### ROCm
| Property | Value |
|----------|-------|
| ROCk Module | Loaded |
| Runtime Version | 1.1 |
| HIP Version | 7.1.52801 |
| Agent Name | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
### Memory
| Property | Value |
|----------|-------|
| System RAM | 61 GiB (shared CPU + GPU) |
| Swap | 8.0 GiB |
| /dev/shm | 31 GiB |
### Storage
| Filesystem | Size | Used | Avail | Use% |
|------------|------|------|-------|------|
| / (nvme0n1p2) | 1.9T | 333G | 1.5T | 19% |
### OS and Software
| Property | Value |
|----------|-------|
| OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
| Kernel | 7.0.0-28-generic |
| Python | 3.14.4 |
## VLLM Relevance
- Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
- 61 GB shared memory allows large models without VRAM constraints.
- Only 8 GB swap. Set `--swap-space` conservatively (4-8 GB).
- ROCm and HIP 7.1 are installed. vLLM ROCm support requires ROCm 6.3+.
- Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. Verify compatibility.
- Radeon 8060S falls under Ryzen AI MAX series (gfx1151/1150). vLLM lists this as supported.
- 1.5 TB free disk space is ample for model storage.
## See Also
- [[VLLM Overview]]
- [[Shared Memory Architecture Dankclaw]]
- [[RAM CPU Offloading in VLLM]]
- [[Running Qwen3.6 27B FP8 on Dankclaw]]