docs(frank-tts): record the NPU spike verdict and the iGPU co-tenancy plan
The overnight spike settled whether frank-tts can run on the Strix Halo NPU. It cannot, and the wall is the loader, not the silicon: AMD's own Linux runtime opens /dev/accel/accel0, succeeds at the GEM ioctls, then declines the NuTTS-Air graph without emitting a single log line — and the one run that did load has never reproduced. The accelerator that does exist is the gfx1151 iGPU behind -DGGML_VULKAN=ON, a flag our own Containerfile.engine writes OFF, and Frank set the target shape to co-tenancy: the engine resident beside frank-dragoon, not a handover. - report/tts-npu-spike-verdict-2026-10-19.md owns the measured result: the exoneration chain, the two arms' real-time bars (12.5 codes/s deployed vs 50.00 for NuTTS-Air), the 65,536-entry speech-token census (ids 151,671-217,206), the host gates (rootless cannot read accel0; dpkg-deb the vendor XRT, never dpkg -i), the iGPU census (18,523 of 116,733 MiB GTT in use, 17 GiB host RAM available, GTT pages are host pages), the two gates that decide co-tenancy (RADV heap for an APU tenant; dragoon's TTFT with a synthesis in flight), and the five-file change surface for the trial. - report/npu-tts-runtime-landscape-2026-10-19.md and report/qwen3-tts-payload-portability-2026-10-19.md are the research slices; both now point at the verdict page instead of ending in loose ends. - The vault page gains "Where the engine computes", so an operator doing a TTS task learns the NPU is a closed door without re-walking it. No fleet change: frank-tts / frank-tts-engine still serve narrator1-3 on CPU. tests: ./tests/run.sh 450/450 green.
This commit is contained in:
1 parent
88342c144b
commit
1e52e45fae
4 files changed
+936
No files matched your search
@@ -0,0 +1,319 @@
|
||||
# XDNA2 NPU inference runtimes on Linux (Strix Halo) — can any of them serve TTS?
|
||||
|
||||
Research slice, 2026-10-19. Worker: `NpuRuntimeLandscape`. Research only, no code.
|
||||
Question: every way to execute model inference on an AMD XDNA2 NPU on Linux as of
|
||||
2026-10, and whether any of them can serve a speech-synthesis model.
|
||||
|
||||
> **Status, same night — the spike that followed this slice settled it.** NO-GO on the
|
||||
> NPU (the loader declines the graph, not the silicon); the actionable door is the gfx1151
|
||||
> iGPU behind `-DGGML_VULKAN=ON`. Measured contract, exoneration chain and host gates:
|
||||
> `tts-npu-spike-verdict-2026-10-19.md`. That page supersedes §8–§9 below.
|
||||
|
||||
Convention: vendor claims carry URL + the version/date they hold for.
|
||||
**[host]** = Dankclaw state verified on the live machine 2026-10-19 (Main), not a
|
||||
vendor claim. `[INFERENCE]` = my deduction from the cited artifacts.
|
||||
|
||||
## 0. Answers to the three questions Main ranked
|
||||
|
||||
**(1) A runtime that already runs TTS on XDNA2 today: it exists at AMD, not on our
|
||||
host.** Ryzen AI Software **1.8.0** (docs `latest` == 1.8.0; released 2026-07-23)
|
||||
lists `neutts-air` and `neutts-nano` as model type **TTS**, variant
|
||||
*full-optimized* (NPU-only) — <https://ryzenai.docs.amd.com/en/latest/llm_list.html>;
|
||||
release note "New TTS model support: neutts-air, neutts-nano" —
|
||||
<https://ryzenai.docs.amd.com/en/latest/relnotes.html>. Pre-compiled artifacts are
|
||||
published: `amd/neutts-air_rai_1.8.0_npu_4K` (arch `qwen2`, lastModified
|
||||
2026-07-23, Apache-2.0, "AWQ / Group 128 / Asymmetric / BFP16 activations / UINT4
|
||||
Weights") and `amd/neutts-nano_rai_1.8.0_npu_4K` (arch `llama`).
|
||||
**Neither artifact is a TTS engine.** Both file lists are LM-only (`model.onnx`,
|
||||
`genai_config.json`, BPE tokenizer, AIE-ML compiler metastate: `dd_metastate_*`
|
||||
`.ctrlpkt/.super/.state`, `cache/txn_bins.zip`) with **no codec, no vocoder, no
|
||||
reference-audio encoder**. [INFERENCE from HF file lists]
|
||||
|
||||
**(2) FastFlowLM: no TTS, and we cannot add one** — see §2. Halogen: no TTS entry
|
||||
**[host]**. So "something installed already serves TTS on the NPU" is FALSE.
|
||||
|
||||
**(3) Our own ONNX graph compiled for the NPU on Linux:** possible for the plain
|
||||
ONNX flows (they compile on Linux), **impossible for a new LLM/OGA model** — model
|
||||
generation is Windows-only in 1.8.0 — and gated by an operator table where BF16
|
||||
lacks Softmax/LayerNorm/ReLU/Gelu (see §4).
|
||||
|
||||
## 1. Landscape: every way to reach the XDNA2 array on Linux
|
||||
|
||||
| # | Path | License | Strix Halo + Linux, 2026-10 | Speech synthesis |
|
||||
|---|------|---------|------------------------------|------------------|
|
||||
| 1 | Ryzen AI SW → ONNX Runtime + Vitis AI / Ryzen AI EP; compile INT8 (`X2`) or BF16 (`VAIML`) ONNX at session creation | AMD proprietary (tarball) | supported: our device ID/rev is in AMD's STX table; compile+run works on Linux (§3, §4) | user-authored ONNX only; no TTS recipe/demo |
|
||||
| 2 | Same EP driven by **OGA** (`onnxruntime-genai`), "full fusion" NPU-only LLM flow; runner = AMD `model_benchmark` / OGA C++ API | AMD + MS OGA | Linux: **vendor-precompiled models only**, no hybrid flow | **neutts-air / neutts-nano = TTS** (LM half) |
|
||||
| 3 | **FastFlowLM** `flm` | MIT + free proprietary kernels | Strix Halo "Supported"; Ubuntu 24.04/25.10/**26.04**, Arch | none (ASR input only) |
|
||||
| 4 | **IRON / mlir-aie** + Peano → hand-written AIE-ML design → XRT/`amdxdna` | Apache-2.0 + LLVM exception | explicitly documented "AMD Ryzen™ AI on Linux"; same driver/XRT stack | anything, hand-written; nothing ships TTS |
|
||||
| 5 | Vendor-curated AIE-ML graph sets (halogen `HALOGEN_NPU_MODELS`, FLM packs) | vendor | present **[host]** | none **[host]** |
|
||||
| 6 | Windows-only: DirectML, Ryzen AI Windows EP (`ryzenai-llm`), Studio Effects, `whispercpp` NPU | — | not usable | (per Lemonade matrix: Windows only) |
|
||||
| 7 | ggml/llama.cpp/**qwentts.cpp**, whisper.cpp, sherpa-onnx, Piper | — | no XDNA backend in ggml/ORT-mainline | CPU / iGPU only |
|
||||
| 8 | IREE, TVM, ONNX Runtime mainline EPs (OpenVINO = Intel NPU, MIGraphX/ROCm = AMD GPU) | — | no XDNA2 target; AMD's NPU EP is only "Xilinx Vitis-AI (preview)" in the ORT EP list and ships inside AMD's own build | n/a |
|
||||
|
||||
Sources: §2–§6 below.
|
||||
|
||||
## 2. FastFlowLM (`/usr/bin/flm` 1.0.1 **[host]**) — catalog + HTTP surface
|
||||
|
||||
* Repo moved into AMD's ROCm org **2026-08-11** (`github.com/ROCm/FastFlowLM`,
|
||||
v1.0.0). "Supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo,
|
||||
Kraken, and Gorgon Point)."
|
||||
<https://github.com/ROCm/FastFlowLM> (README fetched 2026-10-19).
|
||||
* Linux support landed 2026-03; distros: Ubuntu 24.04, 25.10, **26.04**, Arch,
|
||||
generic. Needs NPU firmware ≥ 1.1.0.0, kernel **7.0+** with `amdxdna` (or
|
||||
`amdxdna-dkms`), XRT stack (`ppa:lemonade-team/stable` → `libxrt-npu2`,
|
||||
`amdxdna-dkms`); `flm validate` reads the DRM path (`/dev/accel/accel0`), `flm
|
||||
run` goes through **XRT** and fails with `No such device with index '0'` unless
|
||||
`xrt-smi examine` lists the NPU; `memlock` must be raised.
|
||||
<https://fastflowlm.com/docs/install_lin/> (page dated 2026-03-05, fetched
|
||||
2026-10-19). Static build bundles **XRT v2.21.75** — the same XRT series as the
|
||||
distro package on the host **[host]**.
|
||||
* HTTP surface: `flm serve <model>` → OpenAI-compatible REST, **port 52625**, no
|
||||
auth. Maintained endpoints: `v1/models`, `v1/chat/completions`,
|
||||
`v1/audio/transcriptions`, `v1/embeddings`. **There is no
|
||||
`/v1/audio/speech`.**
|
||||
<https://fastflowlm.com/docs/instructions/server/openapi/>
|
||||
* Catalog 2026-10-19 (<https://fastflowlm.com/docs/models/>): LLaMA, DeepSeek,
|
||||
Qwen, Gemma, MedGemma, TranslateGemma, gpt-oss, LiquidAI/LFM, Phi, Nanbeige,
|
||||
**Whisper — card states "Type: Speech-to-Text (ASR)"**
|
||||
(<https://fastflowlm.com/docs/models/whisper/>), EmbeddingGemma, SmolVLA,
|
||||
Hy-MT2, MiniCPM. No speech-synthesis entry.
|
||||
* Engines are **prebuilt, closed, downloaded from Hugging Face per architecture**
|
||||
("Internet access to HuggingFace is required to download the optimized model
|
||||
kernels"). The only in-tree engine build is Gemma-4E
|
||||
(`cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON`).
|
||||
→ **A new TTS architecture cannot be added to `flm` by us.**
|
||||
|
||||
## 3. AMD Ryzen AI Software 1.8.0 on Linux — install source, device support
|
||||
|
||||
* **Install channel (important correction to the pypi.amd.com evidence).** The pip
|
||||
index is Windows-only for the EP runtime (`ryzenai_dynamic_dispatch-1.7.0-…
|
||||
win_amd64.whl`, `onnxruntime_vitisai-1.23.2-…win_amd64.whl`; `flexmlrt` has a
|
||||
single **linux_aarch64** wheel) — but AMD's Linux stack is not on pip. It ships
|
||||
as a tarball that is, as of 2026-10-19, **publicly reachable without an AMD
|
||||
account**: `https://download.amd.com/opendownload/RyzenAI/1.8.0/ryzen_ai-1.8.0.tgz`
|
||||
→ HTTP 206 on a range probe, `Content-Range: bytes 0-8/**7819506855**`
|
||||
(**≈7.8 GiB**, magic `1f 8b 08` = gzip). (The docs link it behind the
|
||||
EULA form `account.amd.com/...ryzenai-eula-public-xef.html`; the CDN path
|
||||
serves it directly. Verified by me with curl 2026-10-19. I did not finish
|
||||
downloading 7.8 GiB, so the tarball's *contents* list is unverified —
|
||||
`[INFERENCE]` only where noted.)
|
||||
* Driver/XRT package (public, 90,016,137 bytes, `application/zip`):
|
||||
`https://download.amd.com/opendownload/RyzenAI/Driver/RAI_1.8_Linux_NPU_XRT.zip`
|
||||
contains `xrt_202620.2.25.37_24.04-amd64-{base,base-dev,npu}.deb` +
|
||||
`xrt_plugin.2.25.260102.56.release_24.04-amd64-amdxdna.deb` (verified by
|
||||
download + unpack). **The plugin deb ships firmware for
|
||||
`/usr/lib/firmware/amdnpu/17f0_11/npu.dev.sbin`** — exactly the directory our
|
||||
NPU loads **[host]** — plus `1502_00`, `17f0_10`, `17f1_10`, `17f1_13`,
|
||||
`libvxdna.so`, `libxrt_driver_xdna.so.2.25.260102.56`, and the DKMS driver
|
||||
source tarball. **No xclbin/PDI and no compiler binary in this package**: it is
|
||||
driver + XRT shim + firmware only.
|
||||
* **Our device is officially STX.** AMD's APU table
|
||||
(<https://ryzenai.docs.amd.com/en/latest/app_development.md#apu-types>):
|
||||
`0x1022:0x17F0 rev 0x11 → STX` (0x00/0x10 also STX, 0x20 = KRK, `0x1502` rev 0
|
||||
= PHX/HPT). Release notes: "Strix and **Strix Halo** as STX", and the STX/KRK
|
||||
column enables CNN INT8, CNN BF16, NLP BF16, LLM (OGA).
|
||||
* Vendor driver source (same deb, `share/amdxdna/amdxdna.tar.gz`, source dated
|
||||
2026-06-29) maps `0x17f0 rev 0x11 → dev_npu5_info`, `.fw_path =
|
||||
"amdnpu/17f0_11/"`, `default_vbnv = "RyzenAI-npu5"`, and the revision→name table
|
||||
has `{ AIE2_DEV_REVISION_HALO, "NPU Strix Halo" }`; `xrt-smi examine` therefore
|
||||
reports the device as **NPU Strix Halo**.
|
||||
* Linux install recipe (1.8.0 docs
|
||||
<https://ryzenai.docs.amd.com/en/latest/linux.html>): Ubuntu 24.04 packages,
|
||||
**Python 3.12** (`apt install python3.12 python3.12-venv`),
|
||||
`libboost-filesystem1.74.0`, dkms; then `install_ryzen_ai.sh -a yes -p <venv>`;
|
||||
`LD_LIBRARY_PATH=…:${RYZEN_AI_INSTALLATION_PATH}/onnxruntime/lib/`;
|
||||
`quicktest.py` "picks up a simple CNN model, **compiles it** and runs on AMD's
|
||||
Neural Processing Unit" — i.e. **compilation runs on Linux** for the non-LLM
|
||||
flows. "Linux currently supports NPU only flow" (no iGPU EP).
|
||||
Host friction: our box is Ubuntu 26.04 + **Python 3.14** **[host]**, so the venv
|
||||
must be built with a 3.12 interpreter and the packages are 24.04-built
|
||||
`[INFERENCE]`.
|
||||
* LLM flow on Linux
|
||||
(<https://ryzenai.docs.amd.com/en/latest/llm_linux.html>): prequantized
|
||||
ready-to-run models from HF, `deployment/lib`,
|
||||
`RYZENAI_EP_PATH=$PWD/deployment/lib/libonnxruntime_providers_ryzenai.so`,
|
||||
run via the `model_benchmark` binary; **"Linux does not support Hybrid flow"**;
|
||||
**"Model generation is not supported in current release"** (repeated in
|
||||
relnotes 1.8 Known Issues: "Models generated on Windows are compatible with
|
||||
Linux"); needs `memlock unlimited`.
|
||||
Sample throughput in that doc: **49.1 tokens/s decode**, 753 tokens/s prefill
|
||||
(Phi-3.5-mini, 4K model).
|
||||
* Third-party confirmation of the OS split: Lemonade's backend matrix
|
||||
(<https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/README.md>,
|
||||
fetched 2026-10-19) — `flm`/npu/XDNA2 = **Windows, Linux**;
|
||||
`ryzenai-llm`/npu = **Windows only**; `whispercpp`/npu (STT) = **Windows only**;
|
||||
text-to-speech engines `kokoro` and `acestep` = metal / cuda / vulkan / rocm,
|
||||
**no npu device at all**. Lemonade is the "Server Interface" layer of AMD's own
|
||||
Ryzen AI stack (docs index), so this matrix is the vendor-sanctioned view.
|
||||
|
||||
## 4. Can we compile our own graph on Linux? (the no-recipe path)
|
||||
|
||||
Mechanics from <https://ryzenai.docs.amd.com/en/latest/modelrun.md> (1.8.0):
|
||||
ONNX only, opset 17 recommended; the EP compiles on first `InferenceSession` and
|
||||
caches (VitisAI EP cache, or ORT **EP context cache** — `ep.context_enable`, and
|
||||
"precompile the EP context model using Python and deploy it using a C++ program"
|
||||
is explicitly supported); EP auto-partitions unsupported nodes to CPU;
|
||||
BF16 = `target VAIML` (`preferred_data_storage: unvectorized` for transformers),
|
||||
BF16 **STX/KRK-or-newer only**; INT8 = `target=X2` with XINT8/A8W8/A16W8 and
|
||||
"general asymmetric quantization … to enable third-party quantized models";
|
||||
**deployment EP cannot compile BF16 on the fly** — ship a pre-compiled model
|
||||
(<https://ryzenai.docs.amd.com/en/latest/app_development.md>);
|
||||
cache dirs are not portable across EP or NPU-driver versions.
|
||||
|
||||
Operator coverage is the hard gate
|
||||
(<https://ryzenai.docs.amd.com/en/latest/ops_support.md>, "operators supported in
|
||||
Ryzen AI 1.8.0"; "Y" = broad coverage, some configurations may still fail):
|
||||
|
||||
* Good for TTS front-ends/vocoders: `ConvTranspose` **Y in BF16/A16W8/A8W8**
|
||||
(absent only from XINT8), `Conv`, `Resize`, `Upsample`, `Pad`, `LSTM` (BF16),
|
||||
`CumSum` (BF16/XINT8), `STFT` (XINT8 only).
|
||||
* **Blockers for a decoder-style transformer in the plain flows:** in **BF16** the
|
||||
table has **no Softmax, no LayerNormalization, no Relu/Gelu**; LayerNorm is
|
||||
XINT8-only and Softmax is A16W8/A8W8/XINT8-only. So attention either lives in
|
||||
the **LLM/OGA flow** (whose op set is exactly
|
||||
`SimplifiedLayerNormalization`, `SkipSimplifiedLayerNormalization`, `MatMulNBits`
|
||||
(W4ABF16/W4ABFP16), `RotaryEmbedding`, **`GroupQueryAttention`**, `Sigmoid`,
|
||||
`Mul`, `Add`) or must be decomposed into a quantized INT8 flow.
|
||||
* Missing entirely: `GRU`, `Trilu`, `OneHot`, `EyeLike`, `Multinomial`,
|
||||
`NonMaxSuppression`, `CastLike`, `MelWeightMatrix`/`MelFilterGroup`,
|
||||
`RMSNormalization` → graphs that emit them must be rewritten before they
|
||||
partition onto the NPU. [INFERENCE from absence in the table.]
|
||||
* Quantization is a separate tool, not in the driver package: **AMD Quark**
|
||||
(`quark.torch` / `quarkonnx`), which documents `ConvTranspose2d`, `conv1d`,
|
||||
softmax/LayerNorm/BFP quantization (relnotes 1.3/1.2/1.1/0.9) — Quark is *not*
|
||||
on pypi.org under the obvious names **[host probe]** and ships from AMD's
|
||||
gated/pip-private channel.
|
||||
* **New LLM-shaped model (our Qwen-Talker analogue): Windows-only today.**
|
||||
OGA model generation is not available on Linux in 1.8.0 (§3). So a *new*
|
||||
speech-LM cannot be self-served onto the NPU on Linux without AMD (or a
|
||||
Windows box) in the loop.
|
||||
* Practical shape cost: the neutts artifacts carry **one compiled variant per
|
||||
padded sequence length** (128/256/512/768/1024/1536/2048/2304/3072/4096 for
|
||||
prefill; 256…4096 for token-phase attention mask) — i.e. per-shape compile
|
||||
artifacts are the norm for the LLM flow. [INFERENCE from the HF file list.]
|
||||
|
||||
## 5. Does the AMD TTS artifact actually produce speech? (cloned-voice survival)
|
||||
|
||||
NeuTTS is **LM + separate neural codec** ("Simple LM + codec architecture"),
|
||||
clone from ~3 s of reference audio, and the codec is **NeuCodec (50 Hz, single
|
||||
codebook)**: `neuphonic/neucodec` (PyTorch, encode+decode),
|
||||
`neucodec-onnx-decoder{,-int8}` (**decode only**), "Decoder-only (ONNX) codecs
|
||||
require **pre-encoded references** — these ship with the repo for all bundled
|
||||
voices." Phoneme input; 2048-token context ≈ 30 s of audio; NeuTTS-Air ~360 M
|
||||
active / 552 M with embeddings; upstream benchmarks are **CPU-only llama.cpp**
|
||||
(119 tok/s prefill/decode mix on a Ryzen 9 HX 370) and explicitly exclude the
|
||||
codec. <https://raw.githubusercontent.com/neuphonic/neutts/main/README.md>
|
||||
(fetched 2026-10-19); `neuphonic/neutts-air` HF metadata (lastModified
|
||||
2026-08-26, `qwen2`, GGUF + safetensors).
|
||||
|
||||
Consequences for the fleet spike:
|
||||
|
||||
1. AMD's NPU artifact = LM only; **waveform decode and reference encoding are CPU
|
||||
work either way**. Our `reference.wav` voices survive only if we run the
|
||||
NeuCodec **encoder** (PyTorch, per the codec table there is no ONNX encoder)
|
||||
once per voice and then feed pre-encoded codes. That is a one-time offline
|
||||
step per voice; after that the compiled graph takes codes/prompt prefix as
|
||||
ordinary inputs, so nothing about the compiled graph forbids a caller-supplied
|
||||
speaker prefix. `[INFERENCE]`
|
||||
2. Real-time math: 50 Hz codec means **≥50 tokens/s decode** for 1× RT. AMD's own
|
||||
Linux sample output for a 4K Phi-3.5-mini is **49.1 tok/s** — so budget for
|
||||
roughly RT-1× on one 4K "full fusion" graph, before the codec's CPU cost.
|
||||
`[INFERENCE — different model; must be measured.]`
|
||||
3. Host RAM: the same doc reports **peak working set 3.54 GB** for that 4K model.
|
||||
NeuTTS-Air is ~552 M params at W4A-BFP16, so expect a smaller footprint than
|
||||
3.5 GB `[INFERENCE]`; still non-trivial against 14 GiB available **[host]**.
|
||||
4. AMD ships **no TTS demo, runner, or server** on any OS: the examples page has
|
||||
only CNN/BF16/DistilBERT/super-resolution/OGA-LLM/VLM/RAG
|
||||
(<https://ryzenai.docs.amd.com/en/latest/examples.md>); the RyzenAI-SW repo
|
||||
tree (669 paths, `main`, fetched 2026-10-19) contains speech demos **only under
|
||||
`Demos/ASR/{Whisper,Parakeet-TDT}` and `Transformer-examples/ASR/Whisper-AI`**
|
||||
— no TTS directory. Serving TTS therefore means writing our own OGA host
|
||||
(or driving `model_benchmark`-style C++ code) plus phonemizer + NeuCodec glue.
|
||||
|
||||
## 6. Two graphs on one NPU, and the iGPU hazard
|
||||
|
||||
* **Concurrency is a first-class driver feature.** The vendor `amdxdna` source
|
||||
(in `xrt_plugin.2.25.260102.56`, 2026-06-29) sets `.hwctx_limit = 16` for
|
||||
npu4/npu5 — i.e. **16 hardware contexts** on our device — and allocates each
|
||||
context a set of AIE columns with QoS `opc = col_opc (4096) × num_col`
|
||||
(`aie2_ctx.c`, `aie2_solver.h`), with `frame_boundary_preempt` /
|
||||
`fill_preempt_slot` in `aie2_pci.h` (firmware preemption at frame boundaries).
|
||||
So halogen's armed models and a TTS graph can hold separate contexts
|
||||
simultaneously; they contend for columns and OPC share, they are not mutually
|
||||
exclusive.
|
||||
* AMD's user-visible history agrees: 1.4 "Support for up to **16 hardware
|
||||
contexts** on Ryzen AI 300 series"; 1.5 "Application concurrency: improves the
|
||||
resource distribution across applications"; 1.3 "Idle detection and suspension
|
||||
of contexts", "Rebalance feature for AIE hardware resource optimization";
|
||||
1.0 known issue "Support for multiple applications is limited to up to eight"
|
||||
(all in <https://ryzenai.docs.amd.com/en/latest/relnotes.html>).
|
||||
* **Package conflict, the real risk** [INFERENCE]: the vendor stack wants AMD's
|
||||
XRT 2.25.37 + `xrt_plugin…amdxdna` deb **[host: distro `libxrt-npu2
|
||||
1:2.21.75+dfsg-4` installed]**, and firmware must match the loaded `amdxdna.ko`
|
||||
— the vendor README warns that a stale/mismatched `npu.sbin` causes
|
||||
`ERT_CMD_ABORT`/mailbox timeouts, and that a distro **in-tree** `amdxdna` older
|
||||
than the XRT SHIM makes `GET_ARRAY`/`QUERY_TELEMETRY` return
|
||||
`EOPNOTSUPP`/`EINVAL` (<https://github.com/amd/xdna-driver> README,
|
||||
fetched 2026-10-19). Installing AMD's debs over the distro XRT changes the ABI
|
||||
under FastFlowLM/halogen, so the spike must decide the XRT version **once**, for
|
||||
the whole host.
|
||||
* **iGPU / fabric clock:** no vendor document supports a fabric-clock pin, and no
|
||||
vendor document addresses NPU+iGPU concurrency hangs. AMD's Strix Halo
|
||||
optimization doc (ROCm docs, dated 2026-02-20, and its successor "AMD RDNA3.5
|
||||
system optimization", ROCm 10.1.0) covers GPUVM/GART/GTT, the TTM `pages_limit`
|
||||
knob, `amd-ttm`, BIOS VRAM sizing, and minimum kernel versions
|
||||
(<https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimization/strixhalo.html>,
|
||||
<https://rocm.docs.amd.com/en/latest/reference/system-optimization/rdna3-5.html>).
|
||||
The `pp_dpm_fclk` level-7 pin and the GPU+NPU hang observation stay
|
||||
fleet-observed **[host]**, i.e. `[INFERENCE]` territory — no citation exists.
|
||||
|
||||
## 7. Paths I checked and ruled out (so nobody re-walks them)
|
||||
|
||||
* **Windows-only doors**: DirectML; Ryzen AI Windows EP; Lemonade's
|
||||
`ryzenai-llm` and `whispercpp`-NPU engines (matrix above); AMD's OGA model
|
||||
generation. Not usable on Dankclaw.
|
||||
* **Mainline ONNX Runtime** has no AMD-NPU EP: its EP summary lists AMD ROCm and
|
||||
AMD MIGraphX (GPU) and "Xilinx Vitis-AI (*preview*)" only
|
||||
(<https://onnxruntime.ai/docs/execution-providers/>, 2026-10-19); the working
|
||||
EP is AMD's own build (`libonnxruntime_providers_ryzenai.so` in the tarball's
|
||||
deployment dir, per llm_linux).
|
||||
* **ggml / llama.cpp / qwentts.cpp**: no XDNA backend → the current engine
|
||||
(`tts-server` on GGUF, CPU-only build **[host]**) cannot be "turned on" for the
|
||||
NPU; migration means a different runtime, not a flag.
|
||||
* **`flm`**: catalog/engines are closed per-architecture (§2) — the cheapest door
|
||||
by install cost, and it is not a door for TTS.
|
||||
* **IRON / mlir-aie** is the only open toolchain that documents running on the
|
||||
Ryzen AI NPU from Linux ("A close-to-metal Python API for programming AMD Ryzen™
|
||||
AI NPUs, built on an open-source MLIR-based compiler toolchain"; "Getting
|
||||
Started for AMD Ryzen™ AI on Linux", Ubuntu 24.04/24.10, XRT from
|
||||
`ppa:amd-team/xrt`: `libxrt2 libxrt-npu2 … amdxdna-dkms`, `render` group
|
||||
required) — <https://raw.githubusercontent.com/Xilinx/mlir-aie/main/README.md>,
|
||||
fetched 2026-10-19. This is how FLM's kernels were made ("Low-level kernels
|
||||
optimized using IRON + AIE-MLIR", FLM README) — i.e. real, but it is
|
||||
hand-written per-op tile programming, not a model compiler: writing a TTS model
|
||||
here is a multi-month engineering project, not a migration.
|
||||
* **Embedded** Ryzen AI P100 support in the same Linux doc is irrelevant to STX.
|
||||
|
||||
## 8. Recommendation implied by the above (for Main's spike)
|
||||
|
||||
Spike the AMD path as decided, with these constraints planned in: Python 3.12 venv
|
||||
+ 24.04-built wheels on Ubuntu 26.04; one host-wide XRT decision (AMD 2.25.37 vs
|
||||
distro 2.21.75) with firmware/driver pairing checked via `xrt-smi examine` →
|
||||
"NPU Strix Halo"; 7.8 GiB tarball staged off the public CDN URL (§3); pre-encode
|
||||
each `reference.wav` with the **PyTorch** NeuCodec encoder because the ONNX codec
|
||||
is decoder-only; keep phonemizer + codec on CPU; measure decode tokens/s against
|
||||
the 50 Hz budget and peak RSS against the ~14 GiB headroom; expect to write the
|
||||
serving shell yourself (AMD ships `model_benchmark`, no TTS demo).
|
||||
|
||||
## 9. Loose ends (not verified; I stopped on Main's stand-down)
|
||||
|
||||
* Contents of `ryzen_ai-1.8.0.tgz` (7.8 GiB) — whether the Linux tarball contains
|
||||
the AIE-ML compiler (`vaiml`, `aiecompiler_client`) for the *BF16/INT8* flows
|
||||
and which Python wheels it carries. My statement that non-LLM compilation works
|
||||
on Linux rests on AMD's own Linux doc ("quicktest.py … compiles it and runs on
|
||||
… NPU"), not on the package contents. Download was killed mid-flight.
|
||||
* Whether AMD's `neutts` artifacts run correctly under the Linux `model_benchmark`
|
||||
runner (the TTS rows are listed without OS qualifiers; no Linux TTS run is
|
||||
documented anywhere in 1.8.0 docs).
|
||||
* Exact AIE column count/total `ncols` exposed for Strix Halo (driver reads it
|
||||
from AIE metadata: `ndev->total_col = min(XRS_MAX_COL, ndev->aie.metadata.cols)`).
|
||||
Reference in new issue
Block a user