# XDNA2 NPU inference runtimes on Linux (Strix Halo) — can any of them serve TTS? Research slice, 2026-10-19. Worker: `NpuRuntimeLandscape`. Research only, no code. Question: every way to execute model inference on an AMD XDNA2 NPU on Linux as of 2026-10, and whether any of them can serve a speech-synthesis model. > **Status, same night — the spike that followed this slice settled it.** NO-GO on the > NPU (the loader declines the graph, not the silicon); the actionable door is the gfx1151 > iGPU behind `-DGGML_VULKAN=ON`. Measured contract, exoneration chain and host gates: > `tts-npu-spike-verdict-2026-10-19.md`. That page supersedes §8–§9 below. Convention: vendor claims carry URL + the version/date they hold for. **[host]** = Dankclaw state verified on the live machine 2026-10-19 (Main), not a vendor claim. `[INFERENCE]` = my deduction from the cited artifacts. ## 0. Answers to the three questions Main ranked **(1) A runtime that already runs TTS on XDNA2 today: it exists at AMD, not on our host.** Ryzen AI Software **1.8.0** (docs `latest` == 1.8.0; released 2026-07-23) lists `neutts-air` and `neutts-nano` as model type **TTS**, variant *full-optimized* (NPU-only) — ; release note "New TTS model support: neutts-air, neutts-nano" — . Pre-compiled artifacts are published: `amd/neutts-air_rai_1.8.0_npu_4K` (arch `qwen2`, lastModified 2026-07-23, Apache-2.0, "AWQ / Group 128 / Asymmetric / BFP16 activations / UINT4 Weights") and `amd/neutts-nano_rai_1.8.0_npu_4K` (arch `llama`). **Neither artifact is a TTS engine.** Both file lists are LM-only (`model.onnx`, `genai_config.json`, BPE tokenizer, AIE-ML compiler metastate: `dd_metastate_*` `.ctrlpkt/.super/.state`, `cache/txn_bins.zip`) with **no codec, no vocoder, no reference-audio encoder**. [INFERENCE from HF file lists] **(2) FastFlowLM: no TTS, and we cannot add one** — see §2. Halogen: no TTS entry **[host]**. So "something installed already serves TTS on the NPU" is FALSE. **(3) Our own ONNX graph compiled for the NPU on Linux:** possible for the plain ONNX flows (they compile on Linux), **impossible for a new LLM/OGA model** — model generation is Windows-only in 1.8.0 — and gated by an operator table where BF16 lacks Softmax/LayerNorm/ReLU/Gelu (see §4). ## 1. Landscape: every way to reach the XDNA2 array on Linux | # | Path | License | Strix Halo + Linux, 2026-10 | Speech synthesis | |---|------|---------|------------------------------|------------------| | 1 | Ryzen AI SW → ONNX Runtime + Vitis AI / Ryzen AI EP; compile INT8 (`X2`) or BF16 (`VAIML`) ONNX at session creation | AMD proprietary (tarball) | supported: our device ID/rev is in AMD's STX table; compile+run works on Linux (§3, §4) | user-authored ONNX only; no TTS recipe/demo | | 2 | Same EP driven by **OGA** (`onnxruntime-genai`), "full fusion" NPU-only LLM flow; runner = AMD `model_benchmark` / OGA C++ API | AMD + MS OGA | Linux: **vendor-precompiled models only**, no hybrid flow | **neutts-air / neutts-nano = TTS** (LM half) | | 3 | **FastFlowLM** `flm` | MIT + free proprietary kernels | Strix Halo "Supported"; Ubuntu 24.04/25.10/**26.04**, Arch | none (ASR input only) | | 4 | **IRON / mlir-aie** + Peano → hand-written AIE-ML design → XRT/`amdxdna` | Apache-2.0 + LLVM exception | explicitly documented "AMD Ryzen™ AI on Linux"; same driver/XRT stack | anything, hand-written; nothing ships TTS | | 5 | Vendor-curated AIE-ML graph sets (halogen `HALOGEN_NPU_MODELS`, FLM packs) | vendor | present **[host]** | none **[host]** | | 6 | Windows-only: DirectML, Ryzen AI Windows EP (`ryzenai-llm`), Studio Effects, `whispercpp` NPU | — | not usable | (per Lemonade matrix: Windows only) | | 7 | ggml/llama.cpp/**qwentts.cpp**, whisper.cpp, sherpa-onnx, Piper | — | no XDNA backend in ggml/ORT-mainline | CPU / iGPU only | | 8 | IREE, TVM, ONNX Runtime mainline EPs (OpenVINO = Intel NPU, MIGraphX/ROCm = AMD GPU) | — | no XDNA2 target; AMD's NPU EP is only "Xilinx Vitis-AI (preview)" in the ORT EP list and ships inside AMD's own build | n/a | Sources: §2–§6 below. ## 2. FastFlowLM (`/usr/bin/flm` 1.0.1 **[host]**) — catalog + HTTP surface * Repo moved into AMD's ROCm org **2026-08-11** (`github.com/ROCm/FastFlowLM`, v1.0.0). "Supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo, Kraken, and Gorgon Point)." (README fetched 2026-10-19). * Linux support landed 2026-03; distros: Ubuntu 24.04, 25.10, **26.04**, Arch, generic. Needs NPU firmware ≥ 1.1.0.0, kernel **7.0+** with `amdxdna` (or `amdxdna-dkms`), XRT stack (`ppa:lemonade-team/stable` → `libxrt-npu2`, `amdxdna-dkms`); `flm validate` reads the DRM path (`/dev/accel/accel0`), `flm run` goes through **XRT** and fails with `No such device with index '0'` unless `xrt-smi examine` lists the NPU; `memlock` must be raised. (page dated 2026-03-05, fetched 2026-10-19). Static build bundles **XRT v2.21.75** — the same XRT series as the distro package on the host **[host]**. * HTTP surface: `flm serve ` → OpenAI-compatible REST, **port 52625**, no auth. Maintained endpoints: `v1/models`, `v1/chat/completions`, `v1/audio/transcriptions`, `v1/embeddings`. **There is no `/v1/audio/speech`.** * Catalog 2026-10-19 (): LLaMA, DeepSeek, Qwen, Gemma, MedGemma, TranslateGemma, gpt-oss, LiquidAI/LFM, Phi, Nanbeige, **Whisper — card states "Type: Speech-to-Text (ASR)"** (), EmbeddingGemma, SmolVLA, Hy-MT2, MiniCPM. No speech-synthesis entry. * Engines are **prebuilt, closed, downloaded from Hugging Face per architecture** ("Internet access to HuggingFace is required to download the optimized model kernels"). The only in-tree engine build is Gemma-4E (`cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON`). → **A new TTS architecture cannot be added to `flm` by us.** ## 3. AMD Ryzen AI Software 1.8.0 on Linux — install source, device support * **Install channel (important correction to the pypi.amd.com evidence).** The pip index is Windows-only for the EP runtime (`ryzenai_dynamic_dispatch-1.7.0-… win_amd64.whl`, `onnxruntime_vitisai-1.23.2-…win_amd64.whl`; `flexmlrt` has a single **linux_aarch64** wheel) — but AMD's Linux stack is not on pip. It ships as a tarball that is, as of 2026-10-19, **publicly reachable without an AMD account**: `https://download.amd.com/opendownload/RyzenAI/1.8.0/ryzen_ai-1.8.0.tgz` → HTTP 206 on a range probe, `Content-Range: bytes 0-8/**7819506855**` (**≈7.8 GiB**, magic `1f 8b 08` = gzip). (The docs link it behind the EULA form `account.amd.com/...ryzenai-eula-public-xef.html`; the CDN path serves it directly. Verified by me with curl 2026-10-19. I did not finish downloading 7.8 GiB, so the tarball's *contents* list is unverified — `[INFERENCE]` only where noted.) * Driver/XRT package (public, 90,016,137 bytes, `application/zip`): `https://download.amd.com/opendownload/RyzenAI/Driver/RAI_1.8_Linux_NPU_XRT.zip` contains `xrt_202620.2.25.37_24.04-amd64-{base,base-dev,npu}.deb` + `xrt_plugin.2.25.260102.56.release_24.04-amd64-amdxdna.deb` (verified by download + unpack). **The plugin deb ships firmware for `/usr/lib/firmware/amdnpu/17f0_11/npu.dev.sbin`** — exactly the directory our NPU loads **[host]** — plus `1502_00`, `17f0_10`, `17f1_10`, `17f1_13`, `libvxdna.so`, `libxrt_driver_xdna.so.2.25.260102.56`, and the DKMS driver source tarball. **No xclbin/PDI and no compiler binary in this package**: it is driver + XRT shim + firmware only. * **Our device is officially STX.** AMD's APU table (): `0x1022:0x17F0 rev 0x11 → STX` (0x00/0x10 also STX, 0x20 = KRK, `0x1502` rev 0 = PHX/HPT). Release notes: "Strix and **Strix Halo** as STX", and the STX/KRK column enables CNN INT8, CNN BF16, NLP BF16, LLM (OGA). * Vendor driver source (same deb, `share/amdxdna/amdxdna.tar.gz`, source dated 2026-06-29) maps `0x17f0 rev 0x11 → dev_npu5_info`, `.fw_path = "amdnpu/17f0_11/"`, `default_vbnv = "RyzenAI-npu5"`, and the revision→name table has `{ AIE2_DEV_REVISION_HALO, "NPU Strix Halo" }`; `xrt-smi examine` therefore reports the device as **NPU Strix Halo**. * Linux install recipe (1.8.0 docs ): Ubuntu 24.04 packages, **Python 3.12** (`apt install python3.12 python3.12-venv`), `libboost-filesystem1.74.0`, dkms; then `install_ryzen_ai.sh -a yes -p `; `LD_LIBRARY_PATH=…:${RYZEN_AI_INSTALLATION_PATH}/onnxruntime/lib/`; `quicktest.py` "picks up a simple CNN model, **compiles it** and runs on AMD's Neural Processing Unit" — i.e. **compilation runs on Linux** for the non-LLM flows. "Linux currently supports NPU only flow" (no iGPU EP). Host friction: our box is Ubuntu 26.04 + **Python 3.14** **[host]**, so the venv must be built with a 3.12 interpreter and the packages are 24.04-built `[INFERENCE]`. * LLM flow on Linux (): prequantized ready-to-run models from HF, `deployment/lib`, `RYZENAI_EP_PATH=$PWD/deployment/lib/libonnxruntime_providers_ryzenai.so`, run via the `model_benchmark` binary; **"Linux does not support Hybrid flow"**; **"Model generation is not supported in current release"** (repeated in relnotes 1.8 Known Issues: "Models generated on Windows are compatible with Linux"); needs `memlock unlimited`. Sample throughput in that doc: **49.1 tokens/s decode**, 753 tokens/s prefill (Phi-3.5-mini, 4K model). * Third-party confirmation of the OS split: Lemonade's backend matrix (, fetched 2026-10-19) — `flm`/npu/XDNA2 = **Windows, Linux**; `ryzenai-llm`/npu = **Windows only**; `whispercpp`/npu (STT) = **Windows only**; text-to-speech engines `kokoro` and `acestep` = metal / cuda / vulkan / rocm, **no npu device at all**. Lemonade is the "Server Interface" layer of AMD's own Ryzen AI stack (docs index), so this matrix is the vendor-sanctioned view. ## 4. Can we compile our own graph on Linux? (the no-recipe path) Mechanics from (1.8.0): ONNX only, opset 17 recommended; the EP compiles on first `InferenceSession` and caches (VitisAI EP cache, or ORT **EP context cache** — `ep.context_enable`, and "precompile the EP context model using Python and deploy it using a C++ program" is explicitly supported); EP auto-partitions unsupported nodes to CPU; BF16 = `target VAIML` (`preferred_data_storage: unvectorized` for transformers), BF16 **STX/KRK-or-newer only**; INT8 = `target=X2` with XINT8/A8W8/A16W8 and "general asymmetric quantization … to enable third-party quantized models"; **deployment EP cannot compile BF16 on the fly** — ship a pre-compiled model (); cache dirs are not portable across EP or NPU-driver versions. Operator coverage is the hard gate (, "operators supported in Ryzen AI 1.8.0"; "Y" = broad coverage, some configurations may still fail): * Good for TTS front-ends/vocoders: `ConvTranspose` **Y in BF16/A16W8/A8W8** (absent only from XINT8), `Conv`, `Resize`, `Upsample`, `Pad`, `LSTM` (BF16), `CumSum` (BF16/XINT8), `STFT` (XINT8 only). * **Blockers for a decoder-style transformer in the plain flows:** in **BF16** the table has **no Softmax, no LayerNormalization, no Relu/Gelu**; LayerNorm is XINT8-only and Softmax is A16W8/A8W8/XINT8-only. So attention either lives in the **LLM/OGA flow** (whose op set is exactly `SimplifiedLayerNormalization`, `SkipSimplifiedLayerNormalization`, `MatMulNBits` (W4ABF16/W4ABFP16), `RotaryEmbedding`, **`GroupQueryAttention`**, `Sigmoid`, `Mul`, `Add`) or must be decomposed into a quantized INT8 flow. * Missing entirely: `GRU`, `Trilu`, `OneHot`, `EyeLike`, `Multinomial`, `NonMaxSuppression`, `CastLike`, `MelWeightMatrix`/`MelFilterGroup`, `RMSNormalization` → graphs that emit them must be rewritten before they partition onto the NPU. [INFERENCE from absence in the table.] * Quantization is a separate tool, not in the driver package: **AMD Quark** (`quark.torch` / `quarkonnx`), which documents `ConvTranspose2d`, `conv1d`, softmax/LayerNorm/BFP quantization (relnotes 1.3/1.2/1.1/0.9) — Quark is *not* on pypi.org under the obvious names **[host probe]** and ships from AMD's gated/pip-private channel. * **New LLM-shaped model (our Qwen-Talker analogue): Windows-only today.** OGA model generation is not available on Linux in 1.8.0 (§3). So a *new* speech-LM cannot be self-served onto the NPU on Linux without AMD (or a Windows box) in the loop. * Practical shape cost: the neutts artifacts carry **one compiled variant per padded sequence length** (128/256/512/768/1024/1536/2048/2304/3072/4096 for prefill; 256…4096 for token-phase attention mask) — i.e. per-shape compile artifacts are the norm for the LLM flow. [INFERENCE from the HF file list.] ## 5. Does the AMD TTS artifact actually produce speech? (cloned-voice survival) NeuTTS is **LM + separate neural codec** ("Simple LM + codec architecture"), clone from ~3 s of reference audio, and the codec is **NeuCodec (50 Hz, single codebook)**: `neuphonic/neucodec` (PyTorch, encode+decode), `neucodec-onnx-decoder{,-int8}` (**decode only**), "Decoder-only (ONNX) codecs require **pre-encoded references** — these ship with the repo for all bundled voices." Phoneme input; 2048-token context ≈ 30 s of audio; NeuTTS-Air ~360 M active / 552 M with embeddings; upstream benchmarks are **CPU-only llama.cpp** (119 tok/s prefill/decode mix on a Ryzen 9 HX 370) and explicitly exclude the codec. (fetched 2026-10-19); `neuphonic/neutts-air` HF metadata (lastModified 2026-08-26, `qwen2`, GGUF + safetensors). Consequences for the fleet spike: 1. AMD's NPU artifact = LM only; **waveform decode and reference encoding are CPU work either way**. Our `reference.wav` voices survive only if we run the NeuCodec **encoder** (PyTorch, per the codec table there is no ONNX encoder) once per voice and then feed pre-encoded codes. That is a one-time offline step per voice; after that the compiled graph takes codes/prompt prefix as ordinary inputs, so nothing about the compiled graph forbids a caller-supplied speaker prefix. `[INFERENCE]` 2. Real-time math: 50 Hz codec means **≥50 tokens/s decode** for 1× RT. AMD's own Linux sample output for a 4K Phi-3.5-mini is **49.1 tok/s** — so budget for roughly RT-1× on one 4K "full fusion" graph, before the codec's CPU cost. `[INFERENCE — different model; must be measured.]` 3. Host RAM: the same doc reports **peak working set 3.54 GB** for that 4K model. NeuTTS-Air is ~552 M params at W4A-BFP16, so expect a smaller footprint than 3.5 GB `[INFERENCE]`; still non-trivial against 14 GiB available **[host]**. 4. AMD ships **no TTS demo, runner, or server** on any OS: the examples page has only CNN/BF16/DistilBERT/super-resolution/OGA-LLM/VLM/RAG (); the RyzenAI-SW repo tree (669 paths, `main`, fetched 2026-10-19) contains speech demos **only under `Demos/ASR/{Whisper,Parakeet-TDT}` and `Transformer-examples/ASR/Whisper-AI`** — no TTS directory. Serving TTS therefore means writing our own OGA host (or driving `model_benchmark`-style C++ code) plus phonemizer + NeuCodec glue. ## 6. Two graphs on one NPU, and the iGPU hazard * **Concurrency is a first-class driver feature.** The vendor `amdxdna` source (in `xrt_plugin.2.25.260102.56`, 2026-06-29) sets `.hwctx_limit = 16` for npu4/npu5 — i.e. **16 hardware contexts** on our device — and allocates each context a set of AIE columns with QoS `opc = col_opc (4096) × num_col` (`aie2_ctx.c`, `aie2_solver.h`), with `frame_boundary_preempt` / `fill_preempt_slot` in `aie2_pci.h` (firmware preemption at frame boundaries). So halogen's armed models and a TTS graph can hold separate contexts simultaneously; they contend for columns and OPC share, they are not mutually exclusive. * AMD's user-visible history agrees: 1.4 "Support for up to **16 hardware contexts** on Ryzen AI 300 series"; 1.5 "Application concurrency: improves the resource distribution across applications"; 1.3 "Idle detection and suspension of contexts", "Rebalance feature for AIE hardware resource optimization"; 1.0 known issue "Support for multiple applications is limited to up to eight" (all in ). * **Package conflict, the real risk** [INFERENCE]: the vendor stack wants AMD's XRT 2.25.37 + `xrt_plugin…amdxdna` deb **[host: distro `libxrt-npu2 1:2.21.75+dfsg-4` installed]**, and firmware must match the loaded `amdxdna.ko` — the vendor README warns that a stale/mismatched `npu.sbin` causes `ERT_CMD_ABORT`/mailbox timeouts, and that a distro **in-tree** `amdxdna` older than the XRT SHIM makes `GET_ARRAY`/`QUERY_TELEMETRY` return `EOPNOTSUPP`/`EINVAL` ( README, fetched 2026-10-19). Installing AMD's debs over the distro XRT changes the ABI under FastFlowLM/halogen, so the spike must decide the XRT version **once**, for the whole host. * **iGPU / fabric clock:** no vendor document supports a fabric-clock pin, and no vendor document addresses NPU+iGPU concurrency hangs. AMD's Strix Halo optimization doc (ROCm docs, dated 2026-02-20, and its successor "AMD RDNA3.5 system optimization", ROCm 10.1.0) covers GPUVM/GART/GTT, the TTM `pages_limit` knob, `amd-ttm`, BIOS VRAM sizing, and minimum kernel versions (, ). The `pp_dpm_fclk` level-7 pin and the GPU+NPU hang observation stay fleet-observed **[host]**, i.e. `[INFERENCE]` territory — no citation exists. ## 7. Paths I checked and ruled out (so nobody re-walks them) * **Windows-only doors**: DirectML; Ryzen AI Windows EP; Lemonade's `ryzenai-llm` and `whispercpp`-NPU engines (matrix above); AMD's OGA model generation. Not usable on Dankclaw. * **Mainline ONNX Runtime** has no AMD-NPU EP: its EP summary lists AMD ROCm and AMD MIGraphX (GPU) and "Xilinx Vitis-AI (*preview*)" only (, 2026-10-19); the working EP is AMD's own build (`libonnxruntime_providers_ryzenai.so` in the tarball's deployment dir, per llm_linux). * **ggml / llama.cpp / qwentts.cpp**: no XDNA backend → the current engine (`tts-server` on GGUF, CPU-only build **[host]**) cannot be "turned on" for the NPU; migration means a different runtime, not a flag. * **`flm`**: catalog/engines are closed per-architecture (§2) — the cheapest door by install cost, and it is not a door for TTS. * **IRON / mlir-aie** is the only open toolchain that documents running on the Ryzen AI NPU from Linux ("A close-to-metal Python API for programming AMD Ryzen™ AI NPUs, built on an open-source MLIR-based compiler toolchain"; "Getting Started for AMD Ryzen™ AI on Linux", Ubuntu 24.04/24.10, XRT from `ppa:amd-team/xrt`: `libxrt2 libxrt-npu2 … amdxdna-dkms`, `render` group required) — , fetched 2026-10-19. This is how FLM's kernels were made ("Low-level kernels optimized using IRON + AIE-MLIR", FLM README) — i.e. real, but it is hand-written per-op tile programming, not a model compiler: writing a TTS model here is a multi-month engineering project, not a migration. * **Embedded** Ryzen AI P100 support in the same Linux doc is irrelevant to STX. ## 8. Recommendation implied by the above (for Main's spike) Spike the AMD path as decided, with these constraints planned in: Python 3.12 venv + 24.04-built wheels on Ubuntu 26.04; one host-wide XRT decision (AMD 2.25.37 vs distro 2.21.75) with firmware/driver pairing checked via `xrt-smi examine` → "NPU Strix Halo"; 7.8 GiB tarball staged off the public CDN URL (§3); pre-encode each `reference.wav` with the **PyTorch** NeuCodec encoder because the ONNX codec is decoder-only; keep phonemizer + codec on CPU; measure decode tokens/s against the 50 Hz budget and peak RSS against the ~14 GiB headroom; expect to write the serving shell yourself (AMD ships `model_benchmark`, no TTS demo). ## 9. Loose ends (not verified; I stopped on Main's stand-down) * Contents of `ryzen_ai-1.8.0.tgz` (7.8 GiB) — whether the Linux tarball contains the AIE-ML compiler (`vaiml`, `aiecompiler_client`) for the *BF16/INT8* flows and which Python wheels it carries. My statement that non-LLM compilation works on Linux rests on AMD's own Linux doc ("quicktest.py … compiles it and runs on … NPU"), not on the package contents. Download was killed mid-flight. * Whether AMD's `neutts` artifacts run correctly under the Linux `model_benchmark` runner (the TTS rows are listed without OS qualifiers; no Linux TTS run is documented anywhere in 1.8.0 docs). * Exact AIE column count/total `ncols` exposed for Strix Halo (driver reads it from AIE metadata: `ndev->total_col = min(XRS_MAX_COL, ndev->aie.metadata.cols)`).