From 7dce375ab4af449a2fb0874bf5073dadb4306255 Mon Sep 17 00:00:00 2001
From: Frank <frank@dankware>
Date: Thu, 6 Aug 2026 22:40:44 -0700
Subject: [PATCH] docs: fix dankclaw specs (30GiB RAM, 96GiB GPU-visible, ROCm
 7.2.4) + add validate.sh

---
 docs/Dankclaw System Specs.md               | 42 +++++++++--------
 docs/Running Qwen3.6 27B FP8 on Dankclaw.md | 30 +++++++++----
 docs/Shared Memory Architecture Dankclaw.md |  6 ++-
 scripts/validate.sh                         | 50 +++++++++++++++++++++
 4 files changed, 100 insertions(+), 28 deletions(-)
 create mode 100755 scripts/validate.sh

diff --git a/docs/Dankclaw System Specs.md b/docs/Dankclaw System Specs.md
index 7f5a8c0..f5e441f 100644
--- a/docs/Dankclaw System Specs.md	
+++ b/docs/Dankclaw System Specs.md	
@@ -2,8 +2,7 @@
 
 ## Server Overview
 
-Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same 61 GB RAM pool.
-rr
+Dankclaw is an AMD APU server for vLLM inference. The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool.
 ## Hardware Specifications
 
 ### CPU
@@ -22,50 +21,57 @@ rr
 | Property | Value |
 |----------|-------|
 | Model | AMD Radeon 8060S (Strix Halo) |
-| PCIe ID | c4:00.0 |
-| Architecture | Strix/Strix Halo |
-| Memory | Unified with system RAM (no separate VRAM) |
+| Device ID | 0x1586 |
+| Architecture | gfx1151 (Strix Halo) |
+| GPU-Visible Memory | 96 GiB (ROCm addressable range) |
+| Physical RAM | 30 GiB (shared CPU + GPU) |
 
 ### ROCm
 
 | Property | Value |
 |----------|-------|
 | ROCk Module | Loaded |
-| Runtime Version | 1.1 |
-| HIP Version | 7.1.52801 |
+| Runtime Version | 1.18 |
+| ROCm Version | 7.2.4 (from AMD repos) |
 | Agent Name | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S |
+| GPU Arch | gfx1151 (Strix Halo) |
+| GPU Global Memory | 96 GiB (overcommitted, physical is 30 GiB) |
 
 ### Memory
 
 | Property | Value |
 |----------|-------|
-| System RAM | 61 GiB (shared CPU + GPU) |
+| System RAM | 30 GiB (shared CPU + GPU) |
+| GPU-Visible (ROCm) | 96 GiB (overcommitted addressable range) |
 | Swap | 8.0 GiB |
-| /dev/shm | 31 GiB |
+| /dev/shm | ~15 GiB (half of RAM) |
 
 ### Storage
 
 | Filesystem | Size | Used | Avail | Use% |
 |------------|------|------|-------|------|
-| / (nvme0n1p2) | 1.9T | 333G | 1.5T | 19% |
+| / (nvme0n1p2) | 1.9T | 391G | 1.4T | 22% |
 
 ### OS and Software
 
 | Property | Value |
 |----------|-------|
 | OS | Ubuntu 26.04 LTS (Resolute Raccoon) |
-| Kernel | 7.0.0-28-generic |
-| Python | 3.14.4 |
+| Kernel | 7.0.0-29-generic |
+| Python | 3.14.4 (system), 3.12.13 (vLLM venv) |
+| vLLM | 0.26.0+rocm723 |
+| uv | 0.12.2 |
 
 ## VLLM Relevance
 
 - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
-- 61 GB shared memory allows large models without VRAM constraints.
-- Only 8 GB swap. Set `--swap-space` conservatively (4-8 GB).
-- ROCm and HIP 7.1 are installed. vLLM ROCm support requires ROCm 6.3+.
-- Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. Verify compatibility.
-- Radeon 8060S falls under Ryzen AI MAX series (gfx1151/1150). vLLM lists this as supported.
-- 1.5 TB free disk space is ample for model storage.
+- ROCm reports 96 GiB GPU memory (overcommitted). Physical RAM is 30 GiB. vLLM sees 96 GiB and will try to use it. Set `--gpu-memory-utilization` conservatively (0.80-0.85) to avoid OOM when the OS needs memory.
+- Only 8 GiB swap. Set `--swap-space` conservatively (4-8 GiB).
+- ROCm 7.2.4 from AMD repos. vLLM ROCm support requires ROCm 6.3+.
+- Python 3.14.4 is very new. vLLM ROCm wheels target Python 3.12. Use a Python 3.12 venv.
+- Radeon 8060S falls under Ryzen AI MAX series (gfx1151). vLLM lists this as supported.
+- 1.4 TB free disk space is ample for model storage.
+- `libmpi_cxx.so.40` is missing from the system. Install `openmpi-bin` or provide a workaround for torch MPI dependency.
 
 ## See Also
 
diff --git a/docs/Running Qwen3.6 27B FP8 on Dankclaw.md b/docs/Running Qwen3.6 27B FP8 on Dankclaw.md
index 4f5f344..d40b577 100644
--- a/docs/Running Qwen3.6 27B FP8 on Dankclaw.md	
+++ b/docs/Running Qwen3.6 27B FP8 on Dankclaw.md	
@@ -26,10 +26,11 @@ Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. It uses 27
 |----------|-------|
 | CPU | AMD RYZEN AI MAX+ 395 w/ Radeon 8060S (Strix Halo) |
 | GPU | Integrated Radeon 8060S (unified memory) |
-| Unified RAM | 61 GB (shared CPU + GPU) |
-| Swap | 8 GB |
-| ROCm/HIP | 7.1 |
-| Python | 3.14.4 |
+| Physical RAM | 30 GiB (shared CPU + GPU) |
+| GPU-Visible (ROCm) | 96 GiB (overcommitted) |
+| Swap | 8 GiB |
+| ROCm | 7.2.4 |
+| Python | 3.12.13 (vLLM venv) |
 
 ## Memory Calculation
 
@@ -42,11 +43,11 @@ Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. It uses 27
 | Overhead | ~2 GB | ~2 GB |
 | **Total (approximate)** | ~58 GB | ~16.5 GB |
 
-The FP8 variant requires approximately 16-17 GB. This fits within the 61 GB unified memory with comfortable headroom for KV cache.
+The FP8 variant requires approximately 16-17 GB. ThinkingCap-FP8 requires ~29 GB. Both fit within the 30 GiB physical RAM, but ROCm reports 96 GiB to vLLM. Set `--gpu-memory-utilization` conservatively (0.80) to avoid OOM.
 
 ## Prerequisites
 
-Install vLLM for ROCm. Version 0.19.0 or later is required for Qwen3.6:
+Install vLLM for ROCm. Version 0.24.0+ is required for Qwen3.6 with speculative decoding:
 
 ```
 uv venv --python 3.12 --seed
@@ -54,13 +55,24 @@ source .venv/bin/activate
 uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/
 ```
 
-Python 3.14.4 is installed on Dankclaw. vLLM ROCm wheels target Python 3.12. Create a virtual environment with Python 3.12 if available. Verify that Python 3.12 is installed:
+Installed: vLLM 0.26.0+rocm723 in Python 3.12.13 venv at `/home/dank/vllm-env`.
+
+### MPI Dependency
+
+Torch requires `libmpi_cxx.so.40`. If missing, install `openmpi-bin`:
 
 ```
-python3.12 --version
+sudo apt install -y openmpi-bin
 ```
 
-If Python 3.12 is not available, install it or use the system Python with `--no-build-isolation`.
+### Environment Variables
+
+Set these for ROCm on Strix Halo:
+
+```
+export HSA_OVERRIDE_GFX_VERSION=11.5.1
+export HF_HOME=/home/dank/.cache/huggingface
+```
 
 ## Environment Variables
 
diff --git a/docs/Shared Memory Architecture Dankclaw.md b/docs/Shared Memory Architecture Dankclaw.md
index 423b661..82f647d 100644
--- a/docs/Shared Memory Architecture Dankclaw.md	
+++ b/docs/Shared Memory Architecture Dankclaw.md	
@@ -28,7 +28,7 @@ In unified memory:
 
 ### ROCm Version
 
-vLLM supports AMD GPUs with ROCm 6.3 or above. Dankclaw should have ROCm 7.0 or later.
+vLLM supports AMD GPUs with ROCm 6.3 or above. Dankclaw has ROCm 7.2.4 from AMD repos.
 
 Check ROCm version:
 
@@ -141,6 +141,10 @@ Unified memory has lower bandwidth than dedicated VRAM:
 | GDDR6 (RTX 4090) | 1 TB/s |
 | LPDDR5x (Strix Halo) | ~170 GB/s |
 
+### ROCm Overcommit
+
+ROCm reports 96 GiB GPU memory on Strix Halo APU. Physical RAM is 30 GiB. The GPU can address more than physical RAM exists. vLLM will see 96 GiB and try to allocate accordingly. Set `--gpu-memory-utilization` to 0.80 or lower to prevent OOM.
+
 Lower bandwidth means slower inference. Plan for lower throughput.
 
 ### Cache Effects
diff --git a/scripts/validate.sh b/scripts/validate.sh
new file mode 100755
index 0000000..3bb2f38
--- /dev/null
+++ b/scripts/validate.sh
@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+# Validation end-to-end test for vLLM on dankclaw
+# Sends a simple prompt and measures total response time
+# Usage: ./scripts/validate.sh [host] [port]
+
+HOST="${1:-dankclaw}"
+PORT="${2:-8000}"
+MODEL="bottlecapai/ThinkingCap-Qwen3.6-27B-FP8"
+
+echo "=== vLLM Validation Test ==="
+echo "Target: ${HOST}:${PORT}"
+echo "Model: ${MODEL}"
+echo ""
+
+START_TIME=$(date +%s%N)
+
+RESPONSE=$(curl -s -w "\n%{http_code}" \
+    "http://${HOST}:${PORT}/v1/chat/completions" \
+    -H "Content-Type: application/json" \
+    -d "{
+        \"model\": \"${MODEL}\",
+        \"messages\": [
+            {\"role\": \"user\", \"content\": \"What is 2 plus 2? Answer in one word.\"}
+        ],
+        \"max_tokens\": 128,
+        \"temperature\": 0.1
+    }")
+
+END_TIME=$(date +%s%N)
+ELAPSED_MS=$(( (END_TIME - START_TIME) / 1000000 ))
+
+HTTP_CODE=$(echo "$RESPONSE" | tail -1)
+BODY=$(echo "$RESPONSE" | sed '$d')
+
+echo "HTTP Status: ${HTTP_CODE}"
+echo "Response Time: ${ELAPSED_MS}ms"
+echo ""
+
+if [ "$HTTP_CODE" = "200" ]; then
+    CONTENT=$(echo "$BODY" | python3 -c "import sys,json; print(json.load(sys.stdin)['choices'][0]['message']['content'].strip())" 2>/dev/null)
+    echo "Model Output: ${CONTENT}"
+    echo ""
+    echo "PASS — Server responded successfully"
+    echo "TIME=${ELAPSED_MS}"
+    exit 0
+else
+    echo "FAIL — Server returned HTTP ${HTTP_CODE}"
+    echo "Response: ${BODY}"
+    exit 1
+fi