57 lines
6.1 KiB
Markdown
57 lines
6.1 KiB
Markdown
---
|
|
name: strix-halo-npu-readiness-census
|
|
description: "Engine-agnostic census of whether an AMD Strix Halo / Ryzen AI host can run anything on its XDNA2 NPU: driver bind, firmware, device perms, IOMMU, distro XRT, missing AIE compiler, fabric clock, RAM headroom — plus the strace test that separates a dead device from a runtime that declines the graph."
|
|
---
|
|
|
|
Use when asked "can we run <engine/model> on the Strix Halo / Ryzen AI NPU?" for a runtime that is **not** halogen (halogen has its own gate + vendor catalog: use `halogen-npu-feasibility-census` for that). This answers only: is the *host* able, and what is missing.
|
|
|
|
## Why it is usually the wrong door first
|
|
|
|
XDNA2 executes **precompiled AIE partition programs** loaded through XRT → `amdxdna`. There is no generic tensor dispatch: an engine built on ggml/llama.cpp (or any GGUF loader) cannot "enable" the NPU — the port must be a re-export + vendor compile, not a flag. So pair this census with the accelerator the same engine *can* select by build flag (Vulkan/HIP on the gfx1151 iGPU) before proposing NPU work. On Dankclaw the iGPU flag was ours to flip all along (`-DGGML_VULKAN=OFF` written out in the fleet's own engine Containerfile) — check the engine's own build file before treating the NPU as the only hardware door.
|
|
|
|
## Census (all read-only, over ssh from the operator box)
|
|
|
|
```bash
|
|
ssh <host> '
|
|
cat /proc/cmdline; ls /sys/kernel/iommu_groups | wc -l # iommu=pt required; groups==0 => IOMMU off => NPU dead
|
|
lsmod | grep -i amdxdna # in-tree driver
|
|
sudo dmesg | grep -iE "xdna|npu" | tail # "Initialized amdxdna_accel_driver <ver> for 0000:c6:00.1"
|
|
lspci -nn | grep -iE "npu|processing" # Strix Halo = 1022:17f0
|
|
ls /sys/bus/pci/drivers/amdxdna # must list the BDF
|
|
ls -l /dev/accel # accel0, root:render 660
|
|
id -nG <deploy-user> | tr " " "\n" | grep -E "render|video" # rootless podman needs render, else --device fails at create
|
|
find /usr/lib/firmware -iname "*npu*" | head # firmware is /usr/lib/firmware/amdnpu/<devid>_<rev>/npu.sbin*.zst — match the PCI rev
|
|
dpkg -l | grep -iE "xrt|xdna"; ls /opt/xilinx/xrt/lib 2>&1 # distro libxrt-npu2 = XDNA2 runtime; vendor /opt/xilinx/xrt usually absent
|
|
apt-cache policy libxrt2 libxrt-npu2 | grep Candidate
|
|
apt-cache search onnxruntime | head # distro ONNX RT exists; the Ryzen AI EP does not
|
|
python3 -c "import onnxruntime" 2>&1 | tail -1 # absence of the EP + of ryzenai-compile is the real blocker
|
|
for c in /sys/class/drm/card*/device; do echo "-- $c"; cat $c/power_dpm_force_performance_level 2>&1; cat $c/pp_dpm_fclk 2>&1 | tr "\n" " "; echo; done
|
|
nproc; free -g | head -2 # RAM headroom + thread count the engine will pick
|
|
'
|
|
```
|
|
|
|
## Reading it
|
|
|
|
- Driver bound + `accel0` + firmware dir + `iommu=pt` ⇒ **kernel side is ready**; do not re-litigate "is the NPU disabled". A missing device node is *not* the modern failure mode.
|
|
- `auto` on `pp_dpm_fclk` / `power_dpm_force_performance_level` ⇒ fabric clock **unheld**. GPU+NPU concurrently with a moving fclk hangs these hosts and corrupts NPU results; any NPU plan must pin the top fclk step (or run in a container holding it) first.
|
|
- No `ryzenai-compile`/ONNX Ryzen AI EP ⇒ nothing on the host can produce a partition program. That is the enablement delta, and it is a toolchain install + a model-specific export/quantize/compile project, not a config change.
|
|
- `free` headroom matters: NPU partitions want a few GB of host RAM each and can log benign `Failed to alloc dev bo memory, ret -28` under pressure.
|
|
|
|
## Device healthy, load still fails? Classify before blaming the hardware
|
|
|
|
Two different failures, one call apart. Run the load under `strace -f -e trace=openat`:
|
|
|
|
- **Never opens `/dev/accel/accel0`** ⇒ pre-device: loader/shim/env (see `ryzenai-npu-oga-linux-run` — the `Generic Failure` in ~2 ms is this class).
|
|
- **Opens `/dev/accel/accel0`, GEM ioctls succeed, then the EP declines** ⇒ post-device: the *runtime* rejects the graph. The device is innocent; stop the hardware investigation and fix the last file it opened (`model.onnx` = graph rejected; `dd_metastate_*` = partition artifact problem).
|
|
|
|
Ground truth that the device itself is sound, inside the container: `xrt-smi examine` (expect `RyzenAI-npu5 | aie2p | 6x8` + firmware version) then `xrt-smi validate` — `PASSED` incl. `throughput`. Needs `libboost-filesystem1.83.0` + `libboost-program-options1.83.0` (**hyphenated** Ubuntu names, after `apt update`); boost is required by `xrt-smi` only, never by the OGA/EP path. Vendor `quicktest` printing success does **not** prove NPU residency (that build falls back to CPU silently) — sample `/sys/class/accel/accel0/device/mem_usage` during generation instead.
|
|
|
|
When the vendor runtime declines silently, exhaust its own levers and then say so: `GENAI_ORT_VERBOSE_LOGGING`, `GENAI_LOG_ORT_LIB`, `set_log_options`, `set_log_callback(callable)`, `RYZENAI_EP_TRACING`, `RYZENAI_LLM_PROFILE_INIT` all produced zero lines on the 2026-10-19 NuTTS-Air decline. Rule out shadowed artifacts by moving hardlinked mirror copies out of the scanned tree (glob-based sibling scans) — if the failure is byte-identical afterwards, that theory is dead. **If a load ever succeeded once and the byte-identical command never reproduces it (files pristine by inode+mtime), that is a NO-GO, not a lead: no rate may be quoted for a run that cannot be repeated.**
|
|
|
|
## Laws / traps
|
|
|
|
- Read state only. Never probe or load the NPU beside a live inference service — stop + mask the service first (fleet benching policy); this host class has a precedent of a concurrent GPU/NPU experiment needing a power cycle.
|
|
- `sudo podman ...` finds nothing (per-user rootless storage); run podman as the deploy user.
|
|
- `/dev/accel/accel0` existing does not mean a context/PASID can be created; with IOMMU off the failure is at context creation, not at the node.
|
|
- Rootless containers cannot read `accel0` (`root:render 0660`) by any documented flag combination — rootful, or a world-writable udev rule.
|