The overnight spike settled whether frank-tts can run on the Strix Halo NPU. It cannot, and the wall is the loader, not the silicon: AMD's own Linux runtime opens /dev/accel/accel0, succeeds at the GEM ioctls, then declines the NuTTS-Air graph without emitting a single log line — and the one run that did load has never reproduced. The accelerator that does exist is the gfx1151 iGPU behind -DGGML_VULKAN=ON, a flag our own Containerfile.engine writes OFF, and Frank set the target shape to co-tenancy: the engine resident beside frank-dragoon, not a handover. - report/tts-npu-spike-verdict-2026-10-19.md owns the measured result: the exoneration chain, the two arms' real-time bars (12.5 codes/s deployed vs 50.00 for NuTTS-Air), the 65,536-entry speech-token census (ids 151,671-217,206), the host gates (rootless cannot read accel0; dpkg-deb the vendor XRT, never dpkg -i), the iGPU census (18,523 of 116,733 MiB GTT in use, 17 GiB host RAM available, GTT pages are host pages), the two gates that decide co-tenancy (RADV heap for an APU tenant; dragoon's TTFT with a synthesis in flight), and the five-file change surface for the trial. - report/npu-tts-runtime-landscape-2026-10-19.md and report/qwen3-tts-payload-portability-2026-10-19.md are the research slices; both now point at the verdict page instead of ending in loose ends. - The vault page gains "Where the engine computes", so an operator doing a TTS task learns the NPU is a closed door without re-walking it. No fleet change: frank-tts / frank-tts-engine still serve narrator1-3 on CPU. tests: ./tests/run.sh 450/450 green.
320 lines
21 KiB
Markdown
320 lines
21 KiB
Markdown
# XDNA2 NPU inference runtimes on Linux (Strix Halo) — can any of them serve TTS?
|
||
|
||
Research slice, 2026-10-19. Worker: `NpuRuntimeLandscape`. Research only, no code.
|
||
Question: every way to execute model inference on an AMD XDNA2 NPU on Linux as of
|
||
2026-10, and whether any of them can serve a speech-synthesis model.
|
||
|
||
> **Status, same night — the spike that followed this slice settled it.** NO-GO on the
|
||
> NPU (the loader declines the graph, not the silicon); the actionable door is the gfx1151
|
||
> iGPU behind `-DGGML_VULKAN=ON`. Measured contract, exoneration chain and host gates:
|
||
> `tts-npu-spike-verdict-2026-10-19.md`. That page supersedes §8–§9 below.
|
||
|
||
Convention: vendor claims carry URL + the version/date they hold for.
|
||
**[host]** = Dankclaw state verified on the live machine 2026-10-19 (Main), not a
|
||
vendor claim. `[INFERENCE]` = my deduction from the cited artifacts.
|
||
|
||
## 0. Answers to the three questions Main ranked
|
||
|
||
**(1) A runtime that already runs TTS on XDNA2 today: it exists at AMD, not on our
|
||
host.** Ryzen AI Software **1.8.0** (docs `latest` == 1.8.0; released 2026-07-23)
|
||
lists `neutts-air` and `neutts-nano` as model type **TTS**, variant
|
||
*full-optimized* (NPU-only) — <https://ryzenai.docs.amd.com/en/latest/llm_list.html>;
|
||
release note "New TTS model support: neutts-air, neutts-nano" —
|
||
<https://ryzenai.docs.amd.com/en/latest/relnotes.html>. Pre-compiled artifacts are
|
||
published: `amd/neutts-air_rai_1.8.0_npu_4K` (arch `qwen2`, lastModified
|
||
2026-07-23, Apache-2.0, "AWQ / Group 128 / Asymmetric / BFP16 activations / UINT4
|
||
Weights") and `amd/neutts-nano_rai_1.8.0_npu_4K` (arch `llama`).
|
||
**Neither artifact is a TTS engine.** Both file lists are LM-only (`model.onnx`,
|
||
`genai_config.json`, BPE tokenizer, AIE-ML compiler metastate: `dd_metastate_*`
|
||
`.ctrlpkt/.super/.state`, `cache/txn_bins.zip`) with **no codec, no vocoder, no
|
||
reference-audio encoder**. [INFERENCE from HF file lists]
|
||
|
||
**(2) FastFlowLM: no TTS, and we cannot add one** — see §2. Halogen: no TTS entry
|
||
**[host]**. So "something installed already serves TTS on the NPU" is FALSE.
|
||
|
||
**(3) Our own ONNX graph compiled for the NPU on Linux:** possible for the plain
|
||
ONNX flows (they compile on Linux), **impossible for a new LLM/OGA model** — model
|
||
generation is Windows-only in 1.8.0 — and gated by an operator table where BF16
|
||
lacks Softmax/LayerNorm/ReLU/Gelu (see §4).
|
||
|
||
## 1. Landscape: every way to reach the XDNA2 array on Linux
|
||
|
||
| # | Path | License | Strix Halo + Linux, 2026-10 | Speech synthesis |
|
||
|---|------|---------|------------------------------|------------------|
|
||
| 1 | Ryzen AI SW → ONNX Runtime + Vitis AI / Ryzen AI EP; compile INT8 (`X2`) or BF16 (`VAIML`) ONNX at session creation | AMD proprietary (tarball) | supported: our device ID/rev is in AMD's STX table; compile+run works on Linux (§3, §4) | user-authored ONNX only; no TTS recipe/demo |
|
||
| 2 | Same EP driven by **OGA** (`onnxruntime-genai`), "full fusion" NPU-only LLM flow; runner = AMD `model_benchmark` / OGA C++ API | AMD + MS OGA | Linux: **vendor-precompiled models only**, no hybrid flow | **neutts-air / neutts-nano = TTS** (LM half) |
|
||
| 3 | **FastFlowLM** `flm` | MIT + free proprietary kernels | Strix Halo "Supported"; Ubuntu 24.04/25.10/**26.04**, Arch | none (ASR input only) |
|
||
| 4 | **IRON / mlir-aie** + Peano → hand-written AIE-ML design → XRT/`amdxdna` | Apache-2.0 + LLVM exception | explicitly documented "AMD Ryzen™ AI on Linux"; same driver/XRT stack | anything, hand-written; nothing ships TTS |
|
||
| 5 | Vendor-curated AIE-ML graph sets (halogen `HALOGEN_NPU_MODELS`, FLM packs) | vendor | present **[host]** | none **[host]** |
|
||
| 6 | Windows-only: DirectML, Ryzen AI Windows EP (`ryzenai-llm`), Studio Effects, `whispercpp` NPU | — | not usable | (per Lemonade matrix: Windows only) |
|
||
| 7 | ggml/llama.cpp/**qwentts.cpp**, whisper.cpp, sherpa-onnx, Piper | — | no XDNA backend in ggml/ORT-mainline | CPU / iGPU only |
|
||
| 8 | IREE, TVM, ONNX Runtime mainline EPs (OpenVINO = Intel NPU, MIGraphX/ROCm = AMD GPU) | — | no XDNA2 target; AMD's NPU EP is only "Xilinx Vitis-AI (preview)" in the ORT EP list and ships inside AMD's own build | n/a |
|
||
|
||
Sources: §2–§6 below.
|
||
|
||
## 2. FastFlowLM (`/usr/bin/flm` 1.0.1 **[host]**) — catalog + HTTP surface
|
||
|
||
* Repo moved into AMD's ROCm org **2026-08-11** (`github.com/ROCm/FastFlowLM`,
|
||
v1.0.0). "Supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo,
|
||
Kraken, and Gorgon Point)."
|
||
<https://github.com/ROCm/FastFlowLM> (README fetched 2026-10-19).
|
||
* Linux support landed 2026-03; distros: Ubuntu 24.04, 25.10, **26.04**, Arch,
|
||
generic. Needs NPU firmware ≥ 1.1.0.0, kernel **7.0+** with `amdxdna` (or
|
||
`amdxdna-dkms`), XRT stack (`ppa:lemonade-team/stable` → `libxrt-npu2`,
|
||
`amdxdna-dkms`); `flm validate` reads the DRM path (`/dev/accel/accel0`), `flm
|
||
run` goes through **XRT** and fails with `No such device with index '0'` unless
|
||
`xrt-smi examine` lists the NPU; `memlock` must be raised.
|
||
<https://fastflowlm.com/docs/install_lin/> (page dated 2026-03-05, fetched
|
||
2026-10-19). Static build bundles **XRT v2.21.75** — the same XRT series as the
|
||
distro package on the host **[host]**.
|
||
* HTTP surface: `flm serve <model>` → OpenAI-compatible REST, **port 52625**, no
|
||
auth. Maintained endpoints: `v1/models`, `v1/chat/completions`,
|
||
`v1/audio/transcriptions`, `v1/embeddings`. **There is no
|
||
`/v1/audio/speech`.**
|
||
<https://fastflowlm.com/docs/instructions/server/openapi/>
|
||
* Catalog 2026-10-19 (<https://fastflowlm.com/docs/models/>): LLaMA, DeepSeek,
|
||
Qwen, Gemma, MedGemma, TranslateGemma, gpt-oss, LiquidAI/LFM, Phi, Nanbeige,
|
||
**Whisper — card states "Type: Speech-to-Text (ASR)"**
|
||
(<https://fastflowlm.com/docs/models/whisper/>), EmbeddingGemma, SmolVLA,
|
||
Hy-MT2, MiniCPM. No speech-synthesis entry.
|
||
* Engines are **prebuilt, closed, downloaded from Hugging Face per architecture**
|
||
("Internet access to HuggingFace is required to download the optimized model
|
||
kernels"). The only in-tree engine build is Gemma-4E
|
||
(`cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON`).
|
||
→ **A new TTS architecture cannot be added to `flm` by us.**
|
||
|
||
## 3. AMD Ryzen AI Software 1.8.0 on Linux — install source, device support
|
||
|
||
* **Install channel (important correction to the pypi.amd.com evidence).** The pip
|
||
index is Windows-only for the EP runtime (`ryzenai_dynamic_dispatch-1.7.0-…
|
||
win_amd64.whl`, `onnxruntime_vitisai-1.23.2-…win_amd64.whl`; `flexmlrt` has a
|
||
single **linux_aarch64** wheel) — but AMD's Linux stack is not on pip. It ships
|
||
as a tarball that is, as of 2026-10-19, **publicly reachable without an AMD
|
||
account**: `https://download.amd.com/opendownload/RyzenAI/1.8.0/ryzen_ai-1.8.0.tgz`
|
||
→ HTTP 206 on a range probe, `Content-Range: bytes 0-8/**7819506855**`
|
||
(**≈7.8 GiB**, magic `1f 8b 08` = gzip). (The docs link it behind the
|
||
EULA form `account.amd.com/...ryzenai-eula-public-xef.html`; the CDN path
|
||
serves it directly. Verified by me with curl 2026-10-19. I did not finish
|
||
downloading 7.8 GiB, so the tarball's *contents* list is unverified —
|
||
`[INFERENCE]` only where noted.)
|
||
* Driver/XRT package (public, 90,016,137 bytes, `application/zip`):
|
||
`https://download.amd.com/opendownload/RyzenAI/Driver/RAI_1.8_Linux_NPU_XRT.zip`
|
||
contains `xrt_202620.2.25.37_24.04-amd64-{base,base-dev,npu}.deb` +
|
||
`xrt_plugin.2.25.260102.56.release_24.04-amd64-amdxdna.deb` (verified by
|
||
download + unpack). **The plugin deb ships firmware for
|
||
`/usr/lib/firmware/amdnpu/17f0_11/npu.dev.sbin`** — exactly the directory our
|
||
NPU loads **[host]** — plus `1502_00`, `17f0_10`, `17f1_10`, `17f1_13`,
|
||
`libvxdna.so`, `libxrt_driver_xdna.so.2.25.260102.56`, and the DKMS driver
|
||
source tarball. **No xclbin/PDI and no compiler binary in this package**: it is
|
||
driver + XRT shim + firmware only.
|
||
* **Our device is officially STX.** AMD's APU table
|
||
(<https://ryzenai.docs.amd.com/en/latest/app_development.md#apu-types>):
|
||
`0x1022:0x17F0 rev 0x11 → STX` (0x00/0x10 also STX, 0x20 = KRK, `0x1502` rev 0
|
||
= PHX/HPT). Release notes: "Strix and **Strix Halo** as STX", and the STX/KRK
|
||
column enables CNN INT8, CNN BF16, NLP BF16, LLM (OGA).
|
||
* Vendor driver source (same deb, `share/amdxdna/amdxdna.tar.gz`, source dated
|
||
2026-06-29) maps `0x17f0 rev 0x11 → dev_npu5_info`, `.fw_path =
|
||
"amdnpu/17f0_11/"`, `default_vbnv = "RyzenAI-npu5"`, and the revision→name table
|
||
has `{ AIE2_DEV_REVISION_HALO, "NPU Strix Halo" }`; `xrt-smi examine` therefore
|
||
reports the device as **NPU Strix Halo**.
|
||
* Linux install recipe (1.8.0 docs
|
||
<https://ryzenai.docs.amd.com/en/latest/linux.html>): Ubuntu 24.04 packages,
|
||
**Python 3.12** (`apt install python3.12 python3.12-venv`),
|
||
`libboost-filesystem1.74.0`, dkms; then `install_ryzen_ai.sh -a yes -p <venv>`;
|
||
`LD_LIBRARY_PATH=…:${RYZEN_AI_INSTALLATION_PATH}/onnxruntime/lib/`;
|
||
`quicktest.py` "picks up a simple CNN model, **compiles it** and runs on AMD's
|
||
Neural Processing Unit" — i.e. **compilation runs on Linux** for the non-LLM
|
||
flows. "Linux currently supports NPU only flow" (no iGPU EP).
|
||
Host friction: our box is Ubuntu 26.04 + **Python 3.14** **[host]**, so the venv
|
||
must be built with a 3.12 interpreter and the packages are 24.04-built
|
||
`[INFERENCE]`.
|
||
* LLM flow on Linux
|
||
(<https://ryzenai.docs.amd.com/en/latest/llm_linux.html>): prequantized
|
||
ready-to-run models from HF, `deployment/lib`,
|
||
`RYZENAI_EP_PATH=$PWD/deployment/lib/libonnxruntime_providers_ryzenai.so`,
|
||
run via the `model_benchmark` binary; **"Linux does not support Hybrid flow"**;
|
||
**"Model generation is not supported in current release"** (repeated in
|
||
relnotes 1.8 Known Issues: "Models generated on Windows are compatible with
|
||
Linux"); needs `memlock unlimited`.
|
||
Sample throughput in that doc: **49.1 tokens/s decode**, 753 tokens/s prefill
|
||
(Phi-3.5-mini, 4K model).
|
||
* Third-party confirmation of the OS split: Lemonade's backend matrix
|
||
(<https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/README.md>,
|
||
fetched 2026-10-19) — `flm`/npu/XDNA2 = **Windows, Linux**;
|
||
`ryzenai-llm`/npu = **Windows only**; `whispercpp`/npu (STT) = **Windows only**;
|
||
text-to-speech engines `kokoro` and `acestep` = metal / cuda / vulkan / rocm,
|
||
**no npu device at all**. Lemonade is the "Server Interface" layer of AMD's own
|
||
Ryzen AI stack (docs index), so this matrix is the vendor-sanctioned view.
|
||
|
||
## 4. Can we compile our own graph on Linux? (the no-recipe path)
|
||
|
||
Mechanics from <https://ryzenai.docs.amd.com/en/latest/modelrun.md> (1.8.0):
|
||
ONNX only, opset 17 recommended; the EP compiles on first `InferenceSession` and
|
||
caches (VitisAI EP cache, or ORT **EP context cache** — `ep.context_enable`, and
|
||
"precompile the EP context model using Python and deploy it using a C++ program"
|
||
is explicitly supported); EP auto-partitions unsupported nodes to CPU;
|
||
BF16 = `target VAIML` (`preferred_data_storage: unvectorized` for transformers),
|
||
BF16 **STX/KRK-or-newer only**; INT8 = `target=X2` with XINT8/A8W8/A16W8 and
|
||
"general asymmetric quantization … to enable third-party quantized models";
|
||
**deployment EP cannot compile BF16 on the fly** — ship a pre-compiled model
|
||
(<https://ryzenai.docs.amd.com/en/latest/app_development.md>);
|
||
cache dirs are not portable across EP or NPU-driver versions.
|
||
|
||
Operator coverage is the hard gate
|
||
(<https://ryzenai.docs.amd.com/en/latest/ops_support.md>, "operators supported in
|
||
Ryzen AI 1.8.0"; "Y" = broad coverage, some configurations may still fail):
|
||
|
||
* Good for TTS front-ends/vocoders: `ConvTranspose` **Y in BF16/A16W8/A8W8**
|
||
(absent only from XINT8), `Conv`, `Resize`, `Upsample`, `Pad`, `LSTM` (BF16),
|
||
`CumSum` (BF16/XINT8), `STFT` (XINT8 only).
|
||
* **Blockers for a decoder-style transformer in the plain flows:** in **BF16** the
|
||
table has **no Softmax, no LayerNormalization, no Relu/Gelu**; LayerNorm is
|
||
XINT8-only and Softmax is A16W8/A8W8/XINT8-only. So attention either lives in
|
||
the **LLM/OGA flow** (whose op set is exactly
|
||
`SimplifiedLayerNormalization`, `SkipSimplifiedLayerNormalization`, `MatMulNBits`
|
||
(W4ABF16/W4ABFP16), `RotaryEmbedding`, **`GroupQueryAttention`**, `Sigmoid`,
|
||
`Mul`, `Add`) or must be decomposed into a quantized INT8 flow.
|
||
* Missing entirely: `GRU`, `Trilu`, `OneHot`, `EyeLike`, `Multinomial`,
|
||
`NonMaxSuppression`, `CastLike`, `MelWeightMatrix`/`MelFilterGroup`,
|
||
`RMSNormalization` → graphs that emit them must be rewritten before they
|
||
partition onto the NPU. [INFERENCE from absence in the table.]
|
||
* Quantization is a separate tool, not in the driver package: **AMD Quark**
|
||
(`quark.torch` / `quarkonnx`), which documents `ConvTranspose2d`, `conv1d`,
|
||
softmax/LayerNorm/BFP quantization (relnotes 1.3/1.2/1.1/0.9) — Quark is *not*
|
||
on pypi.org under the obvious names **[host probe]** and ships from AMD's
|
||
gated/pip-private channel.
|
||
* **New LLM-shaped model (our Qwen-Talker analogue): Windows-only today.**
|
||
OGA model generation is not available on Linux in 1.8.0 (§3). So a *new*
|
||
speech-LM cannot be self-served onto the NPU on Linux without AMD (or a
|
||
Windows box) in the loop.
|
||
* Practical shape cost: the neutts artifacts carry **one compiled variant per
|
||
padded sequence length** (128/256/512/768/1024/1536/2048/2304/3072/4096 for
|
||
prefill; 256…4096 for token-phase attention mask) — i.e. per-shape compile
|
||
artifacts are the norm for the LLM flow. [INFERENCE from the HF file list.]
|
||
|
||
## 5. Does the AMD TTS artifact actually produce speech? (cloned-voice survival)
|
||
|
||
NeuTTS is **LM + separate neural codec** ("Simple LM + codec architecture"),
|
||
clone from ~3 s of reference audio, and the codec is **NeuCodec (50 Hz, single
|
||
codebook)**: `neuphonic/neucodec` (PyTorch, encode+decode),
|
||
`neucodec-onnx-decoder{,-int8}` (**decode only**), "Decoder-only (ONNX) codecs
|
||
require **pre-encoded references** — these ship with the repo for all bundled
|
||
voices." Phoneme input; 2048-token context ≈ 30 s of audio; NeuTTS-Air ~360 M
|
||
active / 552 M with embeddings; upstream benchmarks are **CPU-only llama.cpp**
|
||
(119 tok/s prefill/decode mix on a Ryzen 9 HX 370) and explicitly exclude the
|
||
codec. <https://raw.githubusercontent.com/neuphonic/neutts/main/README.md>
|
||
(fetched 2026-10-19); `neuphonic/neutts-air` HF metadata (lastModified
|
||
2026-08-26, `qwen2`, GGUF + safetensors).
|
||
|
||
Consequences for the fleet spike:
|
||
|
||
1. AMD's NPU artifact = LM only; **waveform decode and reference encoding are CPU
|
||
work either way**. Our `reference.wav` voices survive only if we run the
|
||
NeuCodec **encoder** (PyTorch, per the codec table there is no ONNX encoder)
|
||
once per voice and then feed pre-encoded codes. That is a one-time offline
|
||
step per voice; after that the compiled graph takes codes/prompt prefix as
|
||
ordinary inputs, so nothing about the compiled graph forbids a caller-supplied
|
||
speaker prefix. `[INFERENCE]`
|
||
2. Real-time math: 50 Hz codec means **≥50 tokens/s decode** for 1× RT. AMD's own
|
||
Linux sample output for a 4K Phi-3.5-mini is **49.1 tok/s** — so budget for
|
||
roughly RT-1× on one 4K "full fusion" graph, before the codec's CPU cost.
|
||
`[INFERENCE — different model; must be measured.]`
|
||
3. Host RAM: the same doc reports **peak working set 3.54 GB** for that 4K model.
|
||
NeuTTS-Air is ~552 M params at W4A-BFP16, so expect a smaller footprint than
|
||
3.5 GB `[INFERENCE]`; still non-trivial against 14 GiB available **[host]**.
|
||
4. AMD ships **no TTS demo, runner, or server** on any OS: the examples page has
|
||
only CNN/BF16/DistilBERT/super-resolution/OGA-LLM/VLM/RAG
|
||
(<https://ryzenai.docs.amd.com/en/latest/examples.md>); the RyzenAI-SW repo
|
||
tree (669 paths, `main`, fetched 2026-10-19) contains speech demos **only under
|
||
`Demos/ASR/{Whisper,Parakeet-TDT}` and `Transformer-examples/ASR/Whisper-AI`**
|
||
— no TTS directory. Serving TTS therefore means writing our own OGA host
|
||
(or driving `model_benchmark`-style C++ code) plus phonemizer + NeuCodec glue.
|
||
|
||
## 6. Two graphs on one NPU, and the iGPU hazard
|
||
|
||
* **Concurrency is a first-class driver feature.** The vendor `amdxdna` source
|
||
(in `xrt_plugin.2.25.260102.56`, 2026-06-29) sets `.hwctx_limit = 16` for
|
||
npu4/npu5 — i.e. **16 hardware contexts** on our device — and allocates each
|
||
context a set of AIE columns with QoS `opc = col_opc (4096) × num_col`
|
||
(`aie2_ctx.c`, `aie2_solver.h`), with `frame_boundary_preempt` /
|
||
`fill_preempt_slot` in `aie2_pci.h` (firmware preemption at frame boundaries).
|
||
So halogen's armed models and a TTS graph can hold separate contexts
|
||
simultaneously; they contend for columns and OPC share, they are not mutually
|
||
exclusive.
|
||
* AMD's user-visible history agrees: 1.4 "Support for up to **16 hardware
|
||
contexts** on Ryzen AI 300 series"; 1.5 "Application concurrency: improves the
|
||
resource distribution across applications"; 1.3 "Idle detection and suspension
|
||
of contexts", "Rebalance feature for AIE hardware resource optimization";
|
||
1.0 known issue "Support for multiple applications is limited to up to eight"
|
||
(all in <https://ryzenai.docs.amd.com/en/latest/relnotes.html>).
|
||
* **Package conflict, the real risk** [INFERENCE]: the vendor stack wants AMD's
|
||
XRT 2.25.37 + `xrt_plugin…amdxdna` deb **[host: distro `libxrt-npu2
|
||
1:2.21.75+dfsg-4` installed]**, and firmware must match the loaded `amdxdna.ko`
|
||
— the vendor README warns that a stale/mismatched `npu.sbin` causes
|
||
`ERT_CMD_ABORT`/mailbox timeouts, and that a distro **in-tree** `amdxdna` older
|
||
than the XRT SHIM makes `GET_ARRAY`/`QUERY_TELEMETRY` return
|
||
`EOPNOTSUPP`/`EINVAL` (<https://github.com/amd/xdna-driver> README,
|
||
fetched 2026-10-19). Installing AMD's debs over the distro XRT changes the ABI
|
||
under FastFlowLM/halogen, so the spike must decide the XRT version **once**, for
|
||
the whole host.
|
||
* **iGPU / fabric clock:** no vendor document supports a fabric-clock pin, and no
|
||
vendor document addresses NPU+iGPU concurrency hangs. AMD's Strix Halo
|
||
optimization doc (ROCm docs, dated 2026-02-20, and its successor "AMD RDNA3.5
|
||
system optimization", ROCm 10.1.0) covers GPUVM/GART/GTT, the TTM `pages_limit`
|
||
knob, `amd-ttm`, BIOS VRAM sizing, and minimum kernel versions
|
||
(<https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimization/strixhalo.html>,
|
||
<https://rocm.docs.amd.com/en/latest/reference/system-optimization/rdna3-5.html>).
|
||
The `pp_dpm_fclk` level-7 pin and the GPU+NPU hang observation stay
|
||
fleet-observed **[host]**, i.e. `[INFERENCE]` territory — no citation exists.
|
||
|
||
## 7. Paths I checked and ruled out (so nobody re-walks them)
|
||
|
||
* **Windows-only doors**: DirectML; Ryzen AI Windows EP; Lemonade's
|
||
`ryzenai-llm` and `whispercpp`-NPU engines (matrix above); AMD's OGA model
|
||
generation. Not usable on Dankclaw.
|
||
* **Mainline ONNX Runtime** has no AMD-NPU EP: its EP summary lists AMD ROCm and
|
||
AMD MIGraphX (GPU) and "Xilinx Vitis-AI (*preview*)" only
|
||
(<https://onnxruntime.ai/docs/execution-providers/>, 2026-10-19); the working
|
||
EP is AMD's own build (`libonnxruntime_providers_ryzenai.so` in the tarball's
|
||
deployment dir, per llm_linux).
|
||
* **ggml / llama.cpp / qwentts.cpp**: no XDNA backend → the current engine
|
||
(`tts-server` on GGUF, CPU-only build **[host]**) cannot be "turned on" for the
|
||
NPU; migration means a different runtime, not a flag.
|
||
* **`flm`**: catalog/engines are closed per-architecture (§2) — the cheapest door
|
||
by install cost, and it is not a door for TTS.
|
||
* **IRON / mlir-aie** is the only open toolchain that documents running on the
|
||
Ryzen AI NPU from Linux ("A close-to-metal Python API for programming AMD Ryzen™
|
||
AI NPUs, built on an open-source MLIR-based compiler toolchain"; "Getting
|
||
Started for AMD Ryzen™ AI on Linux", Ubuntu 24.04/24.10, XRT from
|
||
`ppa:amd-team/xrt`: `libxrt2 libxrt-npu2 … amdxdna-dkms`, `render` group
|
||
required) — <https://raw.githubusercontent.com/Xilinx/mlir-aie/main/README.md>,
|
||
fetched 2026-10-19. This is how FLM's kernels were made ("Low-level kernels
|
||
optimized using IRON + AIE-MLIR", FLM README) — i.e. real, but it is
|
||
hand-written per-op tile programming, not a model compiler: writing a TTS model
|
||
here is a multi-month engineering project, not a migration.
|
||
* **Embedded** Ryzen AI P100 support in the same Linux doc is irrelevant to STX.
|
||
|
||
## 8. Recommendation implied by the above (for Main's spike)
|
||
|
||
Spike the AMD path as decided, with these constraints planned in: Python 3.12 venv
|
||
+ 24.04-built wheels on Ubuntu 26.04; one host-wide XRT decision (AMD 2.25.37 vs
|
||
distro 2.21.75) with firmware/driver pairing checked via `xrt-smi examine` →
|
||
"NPU Strix Halo"; 7.8 GiB tarball staged off the public CDN URL (§3); pre-encode
|
||
each `reference.wav` with the **PyTorch** NeuCodec encoder because the ONNX codec
|
||
is decoder-only; keep phonemizer + codec on CPU; measure decode tokens/s against
|
||
the 50 Hz budget and peak RSS against the ~14 GiB headroom; expect to write the
|
||
serving shell yourself (AMD ships `model_benchmark`, no TTS demo).
|
||
|
||
## 9. Loose ends (not verified; I stopped on Main's stand-down)
|
||
|
||
* Contents of `ryzen_ai-1.8.0.tgz` (7.8 GiB) — whether the Linux tarball contains
|
||
the AIE-ML compiler (`vaiml`, `aiecompiler_client`) for the *BF16/INT8* flows
|
||
and which Python wheels it carries. My statement that non-LLM compilation works
|
||
on Linux rests on AMD's own Linux doc ("quicktest.py … compiles it and runs on
|
||
… NPU"), not on the package contents. Download was killed mid-flight.
|
||
* Whether AMD's `neutts` artifacts run correctly under the Linux `model_benchmark`
|
||
runner (the TTS rows are listed without OS qualifiers; no Linux TTS run is
|
||
documented anywhere in 1.8.0 docs).
|
||
* Exact AIE column count/total `ncols` exposed for Strix Halo (driver reads it
|
||
from AIE metadata: `ndev->total_col = min(XRS_MAX_COL, ndev->aie.metadata.cols)`).
|