50 lines
4.3 KiB
Markdown
50 lines
4.3 KiB
Markdown
---
|
|
name: ryzenai-npu-oga-linux-run
|
|
description: "Run an AMD vendor-compiled Ryzen AI (XDNA2) model — OGA/onnxruntime-genai — on a Linux Ryzen AI host with the in-tree amdxdna driver, without installing the vendor driver or touching the host driver; includes the triage for ORT \"Exception during initialization: Generic Failure\"."
|
|
---
|
|
|
|
Run a vendor-compiled Ryzen AI (XDNA2) model on Linux with the **in-tree** amdxdna driver: no vendor kernel module, no `dpkg -i`, host driver untouched.
|
|
|
|
## Preconditions
|
|
- Host: Strix Point / Strix Halo / Krackan (PCI `1022:17f0`), in-tree `amdxdna` bound to the NPU function (`readlink /sys/bus/pci/devices/<bdf>/driver` → `amdxdna`), `/dev/accel/accel0` present.
|
|
- `ryzen_ai-<ver>.tgz` installed into a venv (`install_ryzen_ai.sh -a yes -p <venv>`; it hard-codes `python3.12` — put a uv-managed 3.12 **bin dir** on PATH, never a symlink: a symlinked shim breaks venv creation with `No module named 'encodings'`).
|
|
- Model artifact MUST match the RAI version: `<name>_rai_<ver>_npu_<ctx>`. Other versions load and emit garbage.
|
|
- Vendor XRT userspace: `dpkg-deb -x` the debs from `RAI_<ver>_Linux_NPU_XRT.zip` (base + npu + `xrt_plugin…amdxdna.deb` — the plugin is what ships `libxrt_driver_xdna.so.*`) into a host dir, bind-mount it at `/opt/xilinx/xrt`.
|
|
|
|
## Container run line (the part that is easy to get wrong)
|
|
```
|
|
--device /dev/accel/accel0 # root:render 0660 → rootful, or a world-writable udev rule
|
|
-v <xrt-root>/opt/xilinx/xrt:/opt/xilinx/xrt:ro
|
|
-e RYZEN_AI_INSTALLATION_PATH=<venv> -e XILINX_XRT=/opt/xilinx/xrt
|
|
-e LD_LIBRARY_PATH=/opt/xilinx/xrt/lib:<venv>/onnxruntime/lib:<venv>/deployment/lib
|
|
# cwd MUST be <venv>/deployment/lib — the RyzenAI EP is dlopen'd by bare name and the
|
|
# only place ORT looks is the working directory.
|
|
# Activate the venv explicitly; a login shell (`bash -lc`) re-sources the profile and
|
|
# can put a numpy-less base python first on PATH.
|
|
```
|
|
|
|
## LAW: mounting XRT is not enough — `/opt/xilinx/xrt/lib` must be ON `LD_LIBRARY_PATH`
|
|
The RyzenAI EP reaches the XDNA shim through XRT's own loader. With the tree bind-mounted
|
|
but off the loader path, `xrt::device(0)` throws in `xrt_core::shim_loader`, the RMM
|
|
allocator fails next, and ORT flattens all of it into
|
|
|
|
```
|
|
[E:onnxruntime:, inference_session.cc:2811] Exception during initialization: Generic Failure
|
|
```
|
|
|
|
Signature that distinguishes this from a real device/driver fault: fails in ~2 ms, **never
|
|
opens `/dev/accel/accel0`**, nothing in `dmesg`/`journalctl -k`, no XRT log (the INI logger
|
|
is not built into the vendor build — `XRT_INI_PATH` + `[Common] verbosity` writes nothing).
|
|
|
|
## Triage recipe for "Generic Failure" (in this order — each step is one call)
|
|
1. `strace -f -e trace=openat` → does it open `/dev/accel/accel0`? If **no**, the failure is pre-device: loader/shim, not the driver. Also note the last file it opens (`genai_config.json` = config parsed, then provider setup fails).
|
|
2. `gdb -q -batch -x cmds --args python driver.py` with `catch throw` + `bt 10` + `continue`. Frames `xrt_core::shim_loader::shim_loader` / `get_userpf_device` / `xrt::device::device` then `ryzenai::Allocator::Alloc` ⇒ the shim never loaded ⇒ fix `LD_LIBRARY_PATH`.
|
|
3. Prove it with a two-arm run in ONE container (toggle only the suspect) before writing the law down. Both arms passing ⇒ that variable was never the cause.
|
|
4. Ground truth on device visibility: `xrt-smi examine` → `RyzenAI-npu5 | aie2p | 6x8` + `NPU Firmware Version`. If xrt-smi dies on `libboost_filesystem.so.1.83.0`, install `libboost_filesystem1.83.0`+`libboost_program_options1.83.0` — **boost is only needed by xrt-smi, not by the OGA/EP path** (verified by purging it and still loading the model).
|
|
5. `voe` in site-packages is a *compiler* toolkit (passes/patterns), not a runtime — swapping `LD_LIBRARY_PATH` to `voe/lib` fixes nothing.
|
|
6. Byte-scanning XRT `.so` files for supported PCI IDs is inconclusive (2-byte ID patterns collide constantly) — don't bother.
|
|
|
|
## Distrust "it ran" on this stack
|
|
- Vendor `quicktest` can print success while computing on CPU (missing peano lib ⇒ silent CPU fallback). Prove NPU residency per run: sample `/sys/class/accel/accel0/device/mem_usage` (needs root) during generation, or read the EP's `Actually running on NPU` line.
|
|
- Never report request #1 after a boot as a rate — first pass includes graph/init cost.
|