serverAppConfigs/report/tts-npu-spike-verdict-2026-10-19.md

TTS on the Strix Halo NPU — spike verdict (2026-10-19)

Question: how can the fleet's TTS engine run on hardware rather than CPU? Answer in one line: not on the NPU — the loader, not the silicon, refuses it; the door that exists is the gfx1151 iGPU behind a build flag we already own.

Spike evidence lives in report/npu-tts-runtime-landscape-2026-10-19.md (which runtimes can reach XDNA2 at all) and report/qwen3-tts-payload-portability-2026-10-19.md (what the payload is and where it can be moved). This page owns only the measured spike result and the model-side contract neither slice has. Throwaway work ran in containers under /home/dank/spikes/npu-tts on Dankclaw; nothing in the fleet was changed — frank-tts and frank-tts-engine still serve narrator1|2|3 on CPU.

Legend: [host] live census 2026-10-19 · [spike] measured in the spike containers · [vendor] primary vendor source, with the version it holds for.

1. Verdict

NO-GO on the NPU, and the wall is a loader. The XDNA2 device is healthy by AMD's own instruments; the runtime that owns this model family opens the device and then declines the graph, silently. One run loaded successfully; byte-identical runs never reproduced it, and a path that cannot be reproduced is not a foundation. [spike]

Go on the iGPU, and the lever is in this repo. dankclaw/voiceservice/Containerfile.engine builds qwentts.cpp (commit 5916bf7) with -DGGML_VULKAN=OFF written out deliberately beside CUDA/HIP/METAL/SYCL/RPC — the running container carries libggml-base.so.0.27.0 + libggml-cpu.so.0.27.0 + tts-server and nothing else. So: -DGGML_VULKAN=ON + --device Vulkan0, plus /dev/dri passthrough in the engine quadlet (the ComfyUI pattern: AddDevice + GroupAdd=keep-groups + a world-writable udev rule). The target shape is co-tenancy — the engine resident on the gfx1151 beside frank-dragoon, not a handover. §5 carries the census and both gates: the heap gate is PASSED — RADV's DEVICE_LOCAL heap is 77.33 GiB with 65.16 GiB of budget while dragoon is loaded, so memory is not the constraint — which leaves the contention gate (dragoon's TTFT with a synthesis in flight) and one unproven piece: whether qwentts.cpp's codec ops have Vulkan kernels for every op. [host] [spike]

Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs only 12.5 codes/s (§2), so this is latency headroom and freed cores, not reviving a dead service. The independent Strix Halo measurement in the portability slice ("use Vulkan exclusively"; NPU decode under 1.5 t/s for an 8B) points the same way.

2. The two arms do not share a real-time bar

Arm	Talker	Codec	Codes per second of audio	Bar
Deployed (frank-tts-engine)	qwen-talker-1.7b-base-Q4_K_M.gguf	qwen-tokenizer-12hz-Q8_0.gguf	12.5	~13 tok/s [host]
Spike (amd/neutts-air_rai_1.8.0_npu_4K)	NuTTS-Air, 24-layer Qwen2	NeuCodec 24000 / hop 480	50.00	~50 tok/s [spike]

Measured from the staged NPU artifact — the contract any future arm must satisfy:

Fact	Value
Vocab	genai_config.json vocab_size 217,652; base BPE 151,643; specials 151,644–151,670 [spike]
Speech tokens	65,536 `<
Stop token	`<
Input ceiling	4096 tokens total, input + output [vendor]
Frame rate	50.00 codes/s confirmed across three references (49.96 / 49.99 / 50.00) [spike]
Reference encode	librosa.load(sr=16000, mono=True) → [1,1,T] float → codec.encode_code(audio_or_path=…) (vendor NeuTTS.encode_reference); CPU RTF 0.28–0.38 on 8 threads, so codes are a cacheable one-time cost, never a per-request charge. neucodec also needs torchtune; install both --no-deps --target <dir> so a second torch never shadows the venv's. [spike]
Pre-encoded voices	/spike/logs/codes-narrator{1,2,3}.npy — 994 / 1,249 / 1,250 codes for 19.90 / 24.98 / 25.00 s [spike]

Gap that any arm inherits: the voice packs carry no transcript. manifest.json holds provenance and crop geometry (crop.source_speech_run 98.739–118.636 s of a 22,569 s audiobook) and no text, and Dankclaw has no ASR at all — whisper, whisper.cpp, faster-whisper all absent. NuTTS conditioning needs ref_text; do not invent it. [host]

3. Exoneration chain — why the blame is not the hardware

Every row is an instrument, not an inference.

Probe	Result
xrt-smi examine	[0000:c6:00.1] RyzenAI-npu5 aie2p 6x8, XRT 2.25.37 (vendor extract), NPU fw 1.1.2.65, amdxdna 7.0.0-38-generic [spike]
xrt-smi validate	PASSED, including throughput [spike]
pyxrt.device(0) in-container	constructs — XRT enumerates the device [spike]
strace -f -e trace=openat on the failing load	genai_config.json read → model.onnx opened → libxrt_driver_xdna.so.2 loaded → /dev/accel/accel0 opened O_RDWR → GEM ioctls succeed → EP declines, before touching dd_metastate_* or cache/txn_bins.zip [spike]
dmesg, fuser /dev/accel/accel0	silent since boot, no holder — the decline never reaches the kernel [spike]
Hardlinked mirror dirs of the model tree (rundir*)	retired, no change — the "metastate shadowing" theory is dead [spike]
Every logging lever in that build: GENAI_ORT_VERBOSE_LOGGING, GENAI_LOG_ORT_LIB, set_log_options, set_log_callback(callable), RYZENAI_EP_TRACING, RYZENAI_LLM_PROFILE_INIT	zero lines — the decline path is silent [spike]
Vendor quicktest	Session successfully initialized — proves the XRT/driver/device path, not NPU residency: that build falls back to CPU silently. Prove residency with /sys/class/accel/accel0/device/mem_usage during generation. [spike] [vendor]

No NPU token rate was ever measured, so none is quoted here. The single successful load (13:17, [load] 0.84s, finite logits, XRT banner at generate time) has not been reproduced by the byte-identical command since, with the model files pristine by inode and mtime. [spike]

4. Host gates the spike proved (so they are not re-litigated)

• Rootless containers cannot read /dev/accel/accel0 (root:render 0660): the host gid never enters the userns map. --group-add keep-groups, --userns=keep-id, --userns=keep-id:gid=990, --group-add render, --group-add gid:990 — all fail. Doors: rootful podman (separate storage), or a world-writable udev rule, which the fleet already does for kfd/renderD*. [spike]
• Never dpkg -i the vendor XRT deb — it installs a DKMS amdxdna over the distro driver. dpkg-deb -x the debs and bind-mount the tree at /opt/xilinx/xrt; the plugin deb is what carries libxrt_driver_xdna.so. [spike]
• In-tree amdxdna 0.7.0-38 + iommu=pt is enough for context creation; a missing device node is not the modern failure mode. [host]
• xrt-smi inside a container needs libboost-filesystem1.83.0 + libboost-program-options1.83.0 (hyphenated Ubuntu names, after apt update); boost is needed by xrt-smi only, never by the OGA/EP path. [spike]
• Headroom is tight and it is host RAM, not a separate pool — see §5 for tonight's census. A multi-GB NPU partition plus an AIE compile workspace does not fit beside frank-dragoon. Benign-looking Failed to alloc dev bo memory, ret -28 is that pressure. [host]

5. Target shape: the TTS engine shares the gfx1151 with frank-dragoon

The deployment Frank wants is both resident at once, not a handover. Measured 2026-10-19 (read-only, nothing loaded or unloaded):

Measure	Value	Reading
iGPU GTT in use	18,523 MiB of 116,733 MiB	≈96 GiB of aperture free; this is the real budget (fleet law) [host]
DRM "VRAM" carve-out	2,048 MiB total, 174 MiB used	what KFD/DRM calls dedicated — not the budget [host]
Host RAM	121 GiB total, 17 GiB available, 79 GiB buff/cache	GTT pages are system pages: an iGPU allocation is host RAM consumed, so the two tenants share one pool [host]
TTS engine today	3.84 GiB RSS, CPU	its device-resident replacement lands in that same pool [host]

Two gates, in this order.

1. Heap gate — PASSED. RADV does not hand a container tenant the 2 GiB DRM carve-out. vulkaninfo inside the engine's own base image (ubuntu:24.04 plus mesa-vulkan-drivers vulkan-tools, --device /dev/dri, rootless) sees Radeon 8060S Graphics (RADV GFX1151), driver radv, API 1.4.318, and reports:

   Heap	size	budget with dragoon loaded	flags
   0	38.67 GiB	32.58 GiB	host-visible, not device-local
   1	77.33 GiB	65.16 GiB	DEVICE_LOCAL

   The heaps sum to the ~116 GiB aperture in Mesa's ⅓/⅔ APU split — the same RAM twice: a Vulkan allocation is a GTT mapping, so a budget is a ceiling on mappings, not a pile of free silicon memory. ggml's Vulkan backend asks for DEVICE_LOCAL, which is heap 1, and the TTS engine needs roughly 4 GiB of it. Memory stops being the question. [spike] Probe traps: vulkaninfo --batch is not a flag in this vulkan-tools (it prints the help page and exits 1 — use vulkaninfo --text), and the container has no NVIDIA ICD, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by device name, or an image that grows an nvidia_icd.json silently points TTS at the 3090 slot.

2. Contention gate — PASSED, measured 2026-10-19 after the cutover. With a 16.9 s synthesis occupying the device, frank-dragoon answered a streamed 124-token request at 17 ms to first token and ~35 tok/s decode, against 17 ms and ~33 tok/s for the same request with the card idle. Compute-unit sharing showed up as nothing. One request per arm, no ladder: these are the gate's evidence, not a throughput record, and the engine's own shape lives in Frank TTS Engine.

Change surface — as landed (PR #34, 6248f02; the CPU shape stays reachable as the rollback):

1. Containerfile.engine build stage: add libvulkan-dev glslc spirv-headers. The prediction above (libvulkan-dev glslang-dev) is wrong on two counts and the configure says so: ggml looks for glslc, which Ubuntu's glslang-tools does not ship (it ships glslangValidator), and find_package(SPIRV-Headers CONFIG REQUIRED) needs the distro's spirv-headers, which does carry the CMake config. glslc is its own source package on noble, at 2023.8 — old enough that only GL_KHR_cooperative_matrix compiles, so the coopmat2 / integer-dot / bf16 / micro-float shader families are built out. Suspect that compiler before the device if a Vulkan prefill ever reads as slow.
2. -DGGML_VULKAN=OFF → ON: one word in the written-out flag list.
3. Runtime stage: mesa-vulkan-drivers and libvulkan1 — the ICD and its loader. An image with a loader and no ICD enumerates zero devices.
4. frank-tts-engine.container: AddDevice=/dev/dri + GroupAdd=keep-groups — the DIRECTORY, never the node: on this hybrid box renderD128 is the NVIDIA card and renderD129 the amdgpu iGPU (readlink -f /sys/class/drm/renderD129/device/driver → amdgpu). On this host the udev rule already makes the nodes world-writable, so the group is belt-and-braces, not the gate.
5. Engine argv: --device Vulkan0, reached through QWENTTS_DEVICE in scripts/frank-tts-engine.env. An empty value passes no flag at all — that is the CPU rollback, and it is why the entrypoint's default is an -v test rather than :=, which also fires on a set-but-empty value and would silently undo the rollback.

Gate 1 recipe, already run and PASSED: mesa-vulkan-drivers + vulkan-tools in a throwaway container off the same base, --device /dev/dri, then vulkaninfo --text and read the VkPhysicalDeviceMemoryProperties heap sizes — no model, no load, no compute. The probe trap: vulkaninfo --batch is not a flag in Ubuntu 24.04's vulkan-tools — it prints the help page and exits 1, which reads like a driver failure. Every package involved has a candidate on the host's apt (vulkan-tools 1.4.341, libvulkan-dev 1.4.341, glslang-dev/glslang-tools 16.2.0); the build stage runs on the digest-pinned Ubuntu 24.04 base, whose versions are older but present. Script: /home/dank/spikes/npu-tts/vk_probe.sh on Dankclaw.

Constraints that do not relax for a co-tenant: /dev/dri passthrough needs the ComfyUI pattern (AddDevice + GroupAdd=keep-groups + the world-writable udev rule) because systemd user-unit spawns carry no supplementary groups; the 64 GiB swap at /swap.img exists to protect co-tenants, and paging a KV pool is worse than the OOM it prevents; pp_dpm_fclk reads auto (unheld), which only becomes a hazard if the NPU ever joins.

6. Re-visit condition

Retry the NPU when a Linux loader claims the graph: an onnxruntime-genai-ryzenai build whose RyzenAI adapter takes the NuTTS-Air Full-Fusion graph on RyzenAI-npu5, or that model appearing in a runtime's own supported catalog with a Linux runner. Then the prerequisites are already mapped and priced: the gated NeuCodec encoder for cloned voices, the ref_text transcripts, the 4096-token prompt budget, and 50 codes/s as the bar.