docs: fix dankclaw specs (30GiB RAM, 96GiB GPU-visible, ROCm 7.2.4) + add validate.sh
This commit is contained in:
1 parent
5076ef828f
commit
7dce375ab4
4 files changed
+100
-28
No files matched your search
@@ -28,7 +28,7 @@ In unified memory:
|
||||
|
||||
### ROCm Version
|
||||
|
||||
vLLM supports AMD GPUs with ROCm 6.3 or above. Dankclaw should have ROCm 7.0 or later.
|
||||
vLLM supports AMD GPUs with ROCm 6.3 or above. Dankclaw has ROCm 7.2.4 from AMD repos.
|
||||
|
||||
Check ROCm version:
|
||||
|
||||
@@ -141,6 +141,10 @@ Unified memory has lower bandwidth than dedicated VRAM:
|
||||
| GDDR6 (RTX 4090) | 1 TB/s |
|
||||
| LPDDR5x (Strix Halo) | ~170 GB/s |
|
||||
|
||||
### ROCm Overcommit
|
||||
|
||||
ROCm reports 96 GiB GPU memory on Strix Halo APU. Physical RAM is 30 GiB. The GPU can address more than physical RAM exists. vLLM will see 96 GiB and try to allocate accordingly. Set `--gpu-memory-utilization` to 0.80 or lower to prevent OOM.
|
||||
|
||||
Lower bandwidth means slower inference. Plan for lower throughput.
|
||||
|
||||
### Cache Effects
|
||||
|
||||
Reference in new issue
Block a user