command-center/agent/managed-skills/ryzenai-oga-model-dir-and-eos-override/SKILL.md

name, description

name	description
ryzenai-oga-model-dir-and-eos-override	Debug OGA on XDNA2 when the RyzenAI EP never claims the If-rooted graph (shadow model dirs, bind-mounted configs, missing device), or a TTS/LLM graph stops on the wrong eos token.

OGA on XDNA2: graph not claimed, or generation stops at once

Two independent failure families in the AMD onnxruntime-genai (OGA) + RyzenAI EP stack on Linux XDNA2 (Strix Halo / Ryzen AI Max). Diagnose them in this order — they are separate gates and each has its own instrument.

Gate 0 — is the NPU path alive at all?

Run the vendor's own instrument before touching anything else:

cd /opt/rai/quicktest && RYZEN_AI_INSTALLATION_PATH=/opt/rai python quicktest.py
# want: "Setting environment for STX/KRK" ... "Session successfully initialized" ... "Test Finished"

It finds the device by shelling out to lspci -nn and grepping 1022:17f0, so it also proves lspci is present in the container. Pass ⇒ the in-tree amdxdna driver and the extracted userspace pair correctly; stop blaming the host.

Separate XRT enumeration from OGA's device probe — they are not the same check:

import sys; sys.path.insert(0, "/opt/xilinx/xrt/python")
import pyxrt; dev = pyxrt.device(0)   # constructs => XRT sees the device

pyxrt.device(0) succeeding while OGA says No RyzenAI devices detected means the refusal is OGA's own predicate, inside libonnxruntime-genai.so (the string lives there, next to RyzenAI, max_length_for_kv_cache, library_path). EP-side env levers worth pulling: RYZENAI_EP_TRACING, RYZENAI_EP_PERFORMANCE_COUNTERS, RYZENAI_LLM_ASYNC_INIT, RYZENAI_LLM_PROFILE_INIT.

Gate 1 — invalid model … Node (If) cond_full

This is ONNX Runtime verifying the raw graph, which happens when no provider claimed it: the EP registered nothing and the CPU provider was left holding an If whose branches mismatch. It is never a corrupted model.

LD_DEBUG=files LD_DEBUG_OUTPUT=/tmp/ld python loadprobe.py
grep -a "calling init:.*ryzenai\|undefined symbol" /tmp/ld.*
# observed: libonnxruntime_providers_ryzenai.so loaded, then
#   symbol lookup error: undefined symbol: GetProvider (fatal)

GetProvider is not a dynamic symbol in the ORT cores, the shared-provider stub, or the EP (nm -D shows neither defines nor wants it), so the failure is registration, not an ELF mismatch. Confirm the cores are identical before chasing versions: both shipped libonnxruntime.so.1.27.0 copies hash equal.

Separate the two gates with an explicit global-scope load before importing OGA:

import ctypes
ctypes.CDLL("/opt/rai/deployment/lib/libonnxruntime_providers_ryzenai.so",
            mode=ctypes.RTLD_GLOBAL)
import onnxruntime_genai as og
og.Model(MODEL)   # registration fixed => error MOVES to "No RyzenAI devices detected"

An error that moves proves you cleared one gate and exposes the next. Silence in the first form means the provider list was empty.

Dead ends (measured, do not retry)

• Shadow model dirs never work. Hardlink mirror, symlink mirror, and even a read-only single-file bind mount of a patched genai_config.json over the export's own genai_config.json all produce the same If rejection. The graph is only claimable when loaded from the export's own directory, and there is no config-path override in 0.14.0-dev: the only env names in the shipped binaries are GENAI_LOG_ORT_LIB, GENAI_ORT_VERBOSE_LOGGING, OG_ORT_LIB. Carrying a config change therefore means either editing the download in place or a single-file bind mount over the real dir (-v file.json:/spike/<model>/genai_config.json:ro) — and if the EP still refuses, the gate is not the config.
• Bisect the image before believing "environment drift". Run the same load probe against every local tag; three tags failing identically rules the image out.
• Rule out host state with instruments, not memory: fuser -v /dev/accel/accel0, dmesg | grep -i xdna, stat -c "%i %y" on the model files, df -h, free -g.
• sysfs class is accel, not misc: glob /sys/class/accel*/*. An empty /sys/class/misc/accel* is your glob, not a missing device.
• GENAI_ORT_VERBOSE_LOGGING=1 and og.set_log_options(...) produce nothing useful in this build; set_log_options is a pybind builtin (no inspect.signature) and rejects severity as an int. Don't burn turns trying to make it talk — use LD_DEBUG.
• strace is absent from the RAI image and apt may have no mirror: LD_DEBUG=files is the loader tracer that needs nothing installed.

Gate 2 — generation stops after one token

OGA reads its stop list from the model section: model.eos_token_id. Chat exports carry the chat-turn-close id (e.g. 151645) there, so the model emits it first and the speech/tool budget is never spent — while search has no eos key: writing search.eos_token_id is a parse refusal (Unknown value "eos_token_id"), not a no-op.

• Fix: rewrite model.eos_token_id to only the task's own end token (TTS: the speech-generation-end id), or pass it through set_search_options if the build accepts that key name.
• A sub-millisecond generate_next_token() with finite, high-confidence logits means the graph ran and chose a stop token — it is not evidence of a CPU fallback.
• Verify a token-range census against the export's added_tokens.json, not against a guessed vocabulary size: NeuCodec codes reach 65,536 entries (ids 151671…217206), and a census bounded at 10k silently miscounts every code as a text token.
• A hardlinked run dir is a loaded gun: open(dst, "w") on a hardlink truncates the original through the shared inode. os.unlink the link before writing a patched file. Also skip .git/ and mirror directories when walking an HF checkout.

Rate measurement

Once a load succeeds, measure at ≥300 steps with a host-side %CPU sampler beside the container (ps -C python -o pid=,pcpu=,rss= every 2 s): a device-bound decode waits on accel0 and stays under one core, a silent CPU fallback pegs at least one. Never report request #1 after a boot — graph capture makes it a warmup artifact.