command-center/agent/managed-skills/ryzenai-npu-oga-linux-spike/SKILL.md

name, description

name	description
ryzenai-npu-oga-linux-spike	Run an AMD-precompiled AMD Ryzen AI (XDNA2) ONNX-GenAI model on a Linux Strix Halo host from public artifacts: staging, rootless NPU device-access wall, container recipe, and the NuTTS-air speech-token contract (65,536-code vocab, 50 codes/s real-time line).

Run an AMD-precompiled XDNA2 model (OGA / onnxruntime-genai) on a Linux Strix Halo host (Dankclaw-class) without installing the vendor stack on the host.

Where the bits actually live (both public, no AMD account)

• Runtime tarball: https://download.amd.com/opendownload/RyzenAI/1.8.0/ryzen_ai-1.8.0.tgz (~7.28 GiB). Verify with a ranged GET — first bytes 1f 8b 08.
• NPU driver/XRT zip: https://download.amd.com/opendownload/RyzenAI/Driver/RAI_1.8_Linux_NPU_XRT.zip (90 MB).
• Dead ends (404, do not re-probe): download.amd.com/driver/linux/RAI_1.8_Linux_NPU_XRT.zip, .../opendownload/RyzenAI/ryzen_ai-1.8.0.tgz, www.xilinx.com/bin/public/openDownload?filename=... (301 → 404). The login-walled EULA form is not the gate; the opendownload tree is the real file host.

Tarball has 30 top-level entries: cp312/linux_x86_64 wheels (onnxruntime_genai_ryzenai-0.14.0, onnxruntime_vitisai-1.27.0, vitis_mllib, flexml*, device_essentials_strx, model_generate, voe-1.8.0), plus deployment.tar.gz (contains lib/libonnxruntime-genai.so, lib/libonnxruntime_providers_ryzenai.so, AIE compiler libs), onnxruntime.tar.gz, quicktest.tar.gz (VitisAI-EP install validator), LLM.tar.gz (text-only OGA examples: run_model.py, tokenizer_factory.py), install_ryzen_ai.sh. voe-4.0-linux_x86_64.tar.gz is only vaip_config.json — no TTS xclbins. AMD ships no TTS runner and no TTS example.

Host rules

• Never dpkg -i the vendor XRT. dpkg-deb -x <deb> <dir> and bind-mount <dir>/opt/xilinx/xrt at the same path in the container. The plugin deb would install a DKMS amdxdna that replaces the distro in-tree driver; the in-tree driver on Ubuntu 26.04 is amdxdna 0.7.0 and already loads amdnpu/17f0_11/npu_7.sbin at boot (dmesg | grep amdxdna).
• Rootless containers cannot read /dev/accel/accel0 (root:render 0660): the host gid never enters the userns gid map, so --device yields nobody:nogroup. Verified failing 2026-10-19: --group-add keep-groups, --userns=keep-id, --userns=keep-id:gid=990 (process gets gid 990, node still unreadable), --group-add render, --group-add gid:990 (both error). Options: run rootful (sudo podman, separate storage → rebuild the image rootful, ~1 min), or add a world-writable udev rule for accel0 (the fleet already does exactly this for kfd/renderD* in 99-comfyui-gpu-access.rules, whose header explains why: systemd user-unit spawns carry no supplementary groups).
• Rootless image build trap: the vendor installer's tar -xf preserves recorded uids (95777 etc.) and aborts under set -e with Cannot change ownership ... Invalid argument, silently skipping the deployment/onnxruntime extraction. Patch it in the Containerfile: sed -i 's/tar -xf/tar --no-same-owner -xf/g' install_ryzen_ai.sh.
• Installer usage: ./install_ryzen_ai.sh -a yes -p /opt/rai -c /opt/rai (python 3.12 required; base python:3.12-slim; needs linux-libc-dev zip pciutils patch libgomp1). It builds a venv from the local wheels, reinstalls onnxruntime-vitisai then --no-deps onnxruntime-genai-ryzenai, installs requirements_extras.txt (transformers 4.57.6), patches bin/activate (sets RYZEN_AI_INSTALLATION_PATH=$VIRTUAL_ENV + LD_LIBRARY_PATH into site-packages), and untars LLM/deployment/onnxruntime/voe/quicktest into the cpp path.
• xrt-smi in a container needs libboost-filesystem1.83.0 + libboost-program-options1.83.0 (hyphenated Ubuntu names) and apt update first; boost is only needed by xrt-smi, never by the OGA/EP path.

HF artifact staging

• Gated (401 on API, 403 on git) for a plain account: neuphonic/neucodec (the only NeuCodec encoder), neuphonic/neucodec-onnx-decoder, neuphonic/neucodec-onnx-decoder-int8, neuphonic/neutts-air. amd/neutts-air_rai_1.8.0_npu_4K is public. Public mirrors of the codec exist (eugenehp/neucodec has the full 1.16 GB pytorch_model.bin) — treat a mirror of a gated release as a licensing decision for the operator, not a workaround to take silently.
• Gated-repo loader trap: HF_HUB_OFFLINE=1 + HF_HOME pointed at a cache laid out as hub/models--<org>--<repo>/snapshots/<rev>/… works, but NeuCodec._from_pretrained asserts the repo id is in ["neuphonic/neucodec", "neuphonic/distill-neucodec"] — a local directory path is refused outright, so pass the repo id and satisfy it from the cache.
• github.com/neuphonic/neutts is public and ships samples/{dave,emily,greta,jo}.{pt,txt,wav} = pre-encoded reference codes + transcripts → a spike needs zero gated bytes.
• Auth without leaking the token into ps: build a 600-perm credential file with a shell builtin and point git at it — umask 077; printf 'https://oauth2:%s@huggingface.co\n' "$(cat ~/.cache/huggingface/token)" > ~/.git-credentials-hf then git -c credential.helper="store --file ~/.git-credentials-hf" clone .... Never git clone https://oauth2:$TOKEN@....

NuTTS-air on XDNA2 (the model-side contract)

• The package is an OGA model dir: genai_config.json (type: qwen2, 24 layers, hidden 896, 14 heads / 2 KV, ctx 32768, vocab_size 217,652, provider_options: [{"RyzenAI": {"hybrid_opt_token_backend": "npu"}}]), plus dd_metastate_* AIE artifacts and model.onnx + model.onnx.data.
• Speech tokens live in the text vocab, and the codebook is 65,536 — not 10,001. tokenizer.json added_tokens holds exactly 65,536 <|speech_N|> entries, ids 151,671–217,206 contiguous, over a base BPE vocab of 151,643; specials are 151,644–151,670 (<|SPEECH_GENERATION_END|> = 151,670). The vendor template maps a codec index straight to its id: "".join(f"<|speech_{i}|>" for i in ref_codes). An encoded reference really does contain codes in the 60k range, so a 10k assumption silently truncates the space.
• The graph's own eos_token_id is [151645, 151643] (the chat turn stop) and search.eos_token_ids is unset — pass 151,670 as an extra eos or speech ends after one token.
• Real-time line = 50 speech codes/s — NeuCodec sample_rate 24000 / hop_length 480, measured exactly (49.96–50.00 across three 20–25 s references). Judge any NPU/iGPU TTS number against 50. (Qwen3-TTS, the fleet's deployed arm, is 12.5 codes/s — do not conflate the two bars.)
• Reference encode is cheap and cacheable: librosa.load(sr=16000, mono=True) → [1,1,T] float tensor → codec.encode_code(audio_or_path=…) (mirrors NeuTTS.encode_reference); CPU encoder RTF 0.28–0.38 on 8 threads. neucodec needs torchtune (RotaryPositionalEmbeddings) — install both --no-deps --target <dir> so a second torch never shadows the venv's.
• Build the prompt by importing the vendor function, never by copying the template: sys.path.insert(0, <neutts repo>); from neutts.neutts import NeuTTS, then NeuTTS._apply_chat_template(SimpleNamespace(tokenizer=hf_tok, input_format="text"), ref_codes, ref_text, text, None) — works on a duck-typed self, so no backbone/codec load. Template: user\n<|TEXT_REPLACE|>\nassistant\n<|SPEECH_REPLACE|>; splice TEXT_PROMPT_START + encode(f"{ref_text} {text}") + TEXT_PROMPT_END at TEXT_REPLACE, then truncate at SPEECH_REPLACE and append [SPEECH_GENERATION_START] + encode("".join(f"<|speech_{i}|>")).
• Voice identity = reference codes + transcript (request-time conditioning). No trained artifact exists to preserve; packs holding only reference.wav require the gated encoder once, offline — and a transcript, which a voice pack does not carry (manifest.json holds provenance/crop geometry only) and a host without ASR cannot produce.
• Full-Fusion input ceiling: 4096 tokens total, input + output.

Status caveat (2026-10-19)

Staged artifacts, image build, device-access matrix, prompt contract, reference pre-encode: done and reproducible. Token generation on the NPU: NO-GO, and the loader is the wall. The device is healthy (xrt-smi validate PASSED incl. throughput, RyzenAI-npu5 / aie2p / 6×8, pyxrt.device(0) constructs), and strace shows the failing load reading genai_config.json, opening model.onnx, loading libxrt_driver_xdna.so.2 and opening /dev/accel/accel0 with successful GEM ioctls — then the RyzenAI EP declines the graph before touching any dd_metastate_*, silently: GENAI_ORT_VERBOSE_LOGGING, GENAI_LOG_ORT_LIB, set_log_options, set_log_callback(callable), RYZENAI_EP_TRACING, RYZENAI_LLM_PROFILE_INIT all emit zero lines. Hardlinked mirror dirs of the model tree were ruled out as shadowing. One run loaded and produced finite logits; byte-identical runs never reproduced it. No token rate was ever measured, so quote none, and remember quicktest success ≠ NPU residency (that build falls back to CPU silently — prove residency via /sys/class/accel/accel0/device/mem_usage).