---
name: ryzenai-oga-model-dir-and-eos-override
description: "Debug OGA on XDNA2 when the RyzenAI EP never claims the If-rooted graph (shadow model dirs, bind-mounted configs, missing device), or a TTS/LLM graph stops on the wrong eos token."
---

# OGA on XDNA2: graph not claimed, or generation stops at once

Two independent failure families in the AMD `onnxruntime-genai` (OGA) + RyzenAI EP stack on
Linux XDNA2 (Strix Halo / Ryzen AI Max). Diagnose them in this order — they are separate
gates and each has its own instrument.

## Gate 0 — is the NPU path alive at all?

Run the vendor's own instrument before touching anything else:

```bash
cd /opt/rai/quicktest && RYZEN_AI_INSTALLATION_PATH=/opt/rai python quicktest.py
# want: "Setting environment for STX/KRK" ... "Session successfully initialized" ... "Test Finished"
```

It finds the device by shelling out to `lspci -nn` and grepping `1022:17f0`, so it also
proves `lspci` is present in the container. Pass ⇒ the in-tree `amdxdna` driver and the
extracted userspace pair correctly; stop blaming the host.

Separate **XRT enumeration** from **OGA's device probe** — they are not the same check:

```python
import sys; sys.path.insert(0, "/opt/xilinx/xrt/python")
import pyxrt; dev = pyxrt.device(0)   # constructs => XRT sees the device
```

`pyxrt.device(0)` succeeding while OGA says `No RyzenAI devices detected` means the
refusal is OGA's own predicate, inside `libonnxruntime-genai.so` (the string lives there,
next to `RyzenAI`, `max_length_for_kv_cache`, `library_path`). EP-side env levers worth
pulling: `RYZENAI_EP_TRACING`, `RYZENAI_EP_PERFORMANCE_COUNTERS`, `RYZENAI_LLM_ASYNC_INIT`,
`RYZENAI_LLM_PROFILE_INIT`.

## Gate 1 — `invalid model … Node (If) cond_full`

This is **ONNX Runtime verifying the raw graph**, which happens when no provider claimed
it: the EP registered nothing and the CPU provider was left holding an `If` whose branches
mismatch. It is never a corrupted model.

```bash
LD_DEBUG=files LD_DEBUG_OUTPUT=/tmp/ld python loadprobe.py
grep -a "calling init:.*ryzenai\|undefined symbol" /tmp/ld.*
# observed: libonnxruntime_providers_ryzenai.so loaded, then
#   symbol lookup error: undefined symbol: GetProvider (fatal)
```

`GetProvider` is not a dynamic symbol in the ORT cores, the shared-provider stub, or the
EP (`nm -D` shows neither defines nor wants it), so the failure is registration, not an
ELF mismatch. Confirm the cores are identical before chasing versions: both shipped
`libonnxruntime.so.1.27.0` copies hash equal.

Separate the two gates with an explicit global-scope load before importing OGA:

```python
import ctypes
ctypes.CDLL("/opt/rai/deployment/lib/libonnxruntime_providers_ryzenai.so",
            mode=ctypes.RTLD_GLOBAL)
import onnxruntime_genai as og
og.Model(MODEL)   # registration fixed => error MOVES to "No RyzenAI devices detected"
```

An error that **moves** proves you cleared one gate and exposes the next. Silence in the
first form means the provider list was empty.

### Dead ends (measured, do not retry)

- **Shadow model dirs never work.** Hardlink mirror, symlink mirror, and even a
  read-only single-file bind mount of a patched `genai_config.json` over the export's own
  `genai_config.json` all produce the same `If` rejection. The graph is only claimable
  when loaded from the export's own directory, and there is no config-path override in
  `0.14.0-dev`: the only env names in the shipped binaries are `GENAI_LOG_ORT_LIB`,
  `GENAI_ORT_VERBOSE_LOGGING`, `OG_ORT_LIB`. Carrying a config change therefore means
  either editing the download in place or a **single-file bind mount over the real dir**
  (`-v file.json:/spike/<model>/genai_config.json:ro`) — and if the EP still refuses, the
  gate is not the config.
- **Bisect the image before believing "environment drift".** Run the same load probe
  against every local tag; three tags failing identically rules the image out.
- **Rule out host state with instruments, not memory:** `fuser -v /dev/accel/accel0`,
  `dmesg | grep -i xdna`, `stat -c "%i %y"` on the model files, `df -h`, `free -g`.
- `sysfs` class is `accel`, not `misc`: glob `/sys/class/accel*/*`. An empty
  `/sys/class/misc/accel*` is your glob, not a missing device.
- `GENAI_ORT_VERBOSE_LOGGING=1` and `og.set_log_options(...)` produce nothing useful in
  this build; `set_log_options` is a pybind builtin (no `inspect.signature`) and rejects
  `severity` as an int. Don't burn turns trying to make it talk — use `LD_DEBUG`.
- `strace` is absent from the RAI image and `apt` may have no mirror: `LD_DEBUG=files` is
  the loader tracer that needs nothing installed.

## Gate 2 — generation stops after one token

OGA reads its stop list from the **model** section: `model.eos_token_id`. Chat exports
carry the chat-turn-close id (e.g. 151645) there, so the model emits it first and the
speech/tool budget is never spent — while `search` has **no** eos key: writing
`search.eos_token_id` is a parse refusal (`Unknown value "eos_token_id"`), not a no-op.

- Fix: rewrite `model.eos_token_id` to only the task's own end token (TTS: the
  speech-generation-end id), or pass it through `set_search_options` if the build accepts
  that key name.
- A sub-millisecond `generate_next_token()` **with finite, high-confidence logits** means
  the graph ran and chose a stop token — it is not evidence of a CPU fallback.
- Verify a token-range census against the export's `added_tokens.json`, not against a
  guessed vocabulary size: NeuCodec codes reach 65,536 entries (ids 151671…217206), and a
  census bounded at 10k silently miscounts every code as a text token.
- A hardlinked run dir is a loaded gun: `open(dst, "w")` on a hardlink truncates the
  **original** through the shared inode. `os.unlink` the link before writing a patched
  file. Also skip `.git/` and mirror directories when walking an HF checkout.

## Rate measurement

Once a load succeeds, measure at ≥300 steps with a host-side `%CPU` sampler beside the
container (`ps -C python -o pid=,pcpu=,rss=` every 2 s): a device-bound decode waits on
`accel0` and stays under one core, a silent CPU fallback pegs at least one. Never report
request #1 after a boot — graph capture makes it a warmup artifact.