serverAppConfigs/report/qwen3-tts-payload-portability-2026-10-19.md

# Qwen3-TTS payload + qwentts.cpp runtime: architecture and portability

Research deliverable (no code). Slice: *what the fleet's TTS payload actually is, and how
portable it is off the CPU ggml path.* Companion slice: `NpuRuntimeLandscape` (what NPU
runtimes exist on Dankclaw).

Evidence date: 2026-10-19. Everything below is grounded in:

- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
  (2026-10-16, upstream HEAD on 2026-10-19) and its ggml submodule
  `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16).
- The two GGUF files as they sit on Dankclaw (`/home/dank/models/Qwen3-TTS-GGUF/`), read
  byte-for-byte from the live host.
- Upstream Qwen3-TTS model cards / release material.

Status: COMPLETE for the assigned slice. ANSWER C was confirmed on the live host the same
night — the running `frank-tts-engine` container carries `libggml-base.so.0.27.0` +
`libggml-cpu.so.0.27.0` and nothing else, and our own
`dankclaw/voiceservice/Containerfile.engine` writes `-DGGML_VULKAN=OFF` out explicitly, so
the iGPU change is a flag in a file the fleet owns. Verdict, measured contract and the NPU
exoneration chain: `tts-npu-spike-verdict-2026-10-19.md`.

---

## 1. What the payload is

Two files, both read byte-for-byte off the live host (`/home/dank/models/Qwen3-TTS-GGUF/`,
owned `dank:dank`, mtimes 2026-10-18 18:31), both from
`https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF` (community converter, not Qwen):

| file | bytes | GGUF | tensors | arch string | file_type |
|---|---|---|---|---|---|
| `qwen-talker-1.7b-base-Q4_K_M.gguf` | 1,219,245,248 | v3, kv=64 | 480 | `qwen3-tts` | `Q4_K_M` |
| `qwen-tokenizer-12hz-Q8_0.gguf` | 291,150,624 | v3, kv=51 | 398 | `qwen3-tts-tokenizer` | `Q8_0` |

`general.name` of the talker file is `Qwen3-TTS-12Hz-1.7B-base`; the header sizes add up
to the file sizes (sum of tensor bytes + header + alignment padding), so nothing in these
files is unaccounted for.

### 1.1 Talker file (480 tensors)

Three models in one container, per `docs/ARCHITECTURE.md` §GGUF layout
(qwentts.cpp @ pinned commit) and confirmed by the tensor census:

| prefix | tensors | bytes | what it is |
|---|---|---|---|
| `talker.*` | 316 | 1,061.9 MB | 28-layer Qwen3 decoder (hidden 2048, FFN 6144, 16 Q heads / 8 KV heads, head_dim 128, RoPE θ=1e6, RMS ε=1e-6, ctx 32768), text stream + codec stream summed, `codec_head` → 3072 codebook-0 logits |
| `code_pred.*` | 88 | 114.8 MB | 5-layer Qwen3 MTP head (hidden 1024, FFN 3072, 15 acoustic codebooks × 2048), own KV cache |
| `spk_enc.*` | 76 | 36.6 MB | ECAPA-TDNN speaker encoder (x-vector, base checkpoints only; mel → conv0 → 3 SE-Res2Net → MFA → ASP → 2048-d vector) |

Dtype split (exact tensor byte sums): `q4_K` 250 tensors / 864.1 MB, `q6_K` 17 / 312.0 MB,
`f32` 176 / 25.8 MB, `f16` 37 / 11.4 MB.

Which tensors carry which quant is the load-bearing portability fact:

- `q6_K` (17): `talker.text_embd.weight [2048, 151936]` (255.3 MB), `talker.codec_embd.weight
  [2048, 3072]`, and the 15 `code_pred.codec_embd.<g>.weight [2048, 2048]` embedding tables.
  These are the llama.cpp `Q4_K_M` "embedding/output rows go to Q6_K" rule, applied to
  embedding tables rather than to an `lm_head`.
- `q4_K` (250): every attention/FFN projection of the talker and of the code predictor.
  Distinct matmul shapes: `[2048,2048]`×73, `[2048,1024]`×62, `[2048,6144]`×56,
  `[6144,2048]`×28, `[1024,2048]`×20, `[1024,3072]`×10, `[1024,1024]`×10, `[3072,1024]`×5,
  `[2048,3072]`×2, `[2048,151936]`×1. Every one is 2-D and ≥ 1024 on both axes — good
  GEMM shapes, nothing exotic.
- `f32` (176) / `f16` (37): all of `spk_enc` (conv weights F16 because K=1/3/5 rows are not
  block-alignable, per the converter's `should_quantize` policy in `tools/quantize.cpp` +
  `quantize.sh`; every bias/norm/LayerScale vector F32), plus the talker/predictor norms and
  the `text_proj` biases.

Metadata the runtime actually reads (talker file): `qwen3-tts.num_code_groups=16`,
`talker.vocab_size=3072`, `text_vocab_size=151936`, `text_hidden_size=2048`,
`position_id_per_seconds=13`, `rope.mrope_section=[24,20,20]`, `mrope_interleaved=1`,
the 7 codec special ids (pad 2148 / bos 2149 / eos 2150 / think 2154 / nothink 2155 /
think_bos 2156 / think_eos 2157), `language_names` (10: chinese, english, german, italian,
portuguese, spanish, japanese, korean, french, russian) with their codec ids, the text-side
special ids (im_start 151644, im_end 151645, tts_pad 151671, tts_bos 151672, tts_eos 151673),
the whole `generation_config.json` (do_sample, top_k 50, top_p 1.0, temperature 0.9,
repetition_penalty 1.05, `subtalker_*` mirrors, max_new_tokens 8192), and an embedded
**Qwen2 BPE tokenizer** (`tokenizer.ggml.model=gpt2`, 151,676 tokens, 151,291 merges,
eos 151643) — that BPE block is 5.95 MB of the header, i.e. 49 % of the header is vocabulary.

Two live-file details that contradict the upstream doc text (worth knowing when reading
`docs/ARCHITECTURE.md`: the doc example block is stale against the shipped file):
`mrope_interleaved` is **1** (doc shows `false`), and `spk_enc.sample_rate` is **24000**
(doc says 16000). Doc also says "404 tensors on the 1.7B voice_design build" — the base
build shipped here has 480 because it carries `spk_enc.*`.

### 1.2 Tokenizer/codec file (398 tensors)

One neural audio codec, 12.5 frames/s over 24 kHz mono, hop 1920, 16 codebooks × 2048
entries (1 semantic + 15 acoustic). Dtype split: `f16` 50 / 143.6 MB, `q8_0` 110 / 76.9 MB,
`f32` 238 / 70.7 MB. `tok_dec.*` 253 tensors / 201.3 MB, `tok_enc.*` 145 / 89.9 MB.

Composition (metadata + tensor census, both files agree):

- **Encoder** (used only to make voice references, not on the synthesis hot path): SEANet
  conv stack (init k=7, 4 stages with ratios 8/6/5/4 → cumulative 960×, 64→512 ch, final
  k=3), an 8-layer transformer (hidden 512, 8 heads, head_dim 64, FFN 2048, RoPE 1e4,
  LayerNorm+bias, GELU, LayerScale), a downsample conv k=4 s=2 (25 → 12.5 Hz), then the RVQ
  argmin loop. Metadata records `encoder.num_quantizers=32` with
  `encoder_valid_num_quantizers=16` — 16 codebooks are used, 16 are dead weight carried from
  the upstream config.
- **Decoder** (the hot path: every generated frame): 16 codebook lookups + per-split
  `output_proj` (256→512), `conv_pre`, an 8-layer transformer (hidden 512, 16/16 heads,
  head_dim 64, FFN 1024, **sliding window 72**, RMSNorm, SwiGLU, LayerScale, input_proj
  1024→512 / output_proj 512→1024), a ConvNeXt upampler (`upsampling_ratios=[2,2]`, causal
  transposed convs + depthwise k=7), then a DAC-style decoder `dec.0..3`
  (`upsample_rates=[8,5,4,3]`, channels 1536→768→384→192→96, `decoder_dim=1536`) whose blocks
  are SnakeBeta → causal transposed conv (k = 2·stride) → 3 residual units (dilations 1/3/9)
  → snake_post → conv_post k=7 → 96→1.
- All 32 codebook tensors are **F32, 67.1 MB total** (16 encode + 16 decode, `[256, 2048]`),
  deliberately unquantized: the converter's `should_quantize` keeps RVQ codebooks and their
  wrapping projections at F32 because nearest-neighbour lookup is sensitive to per-row
  quantisation noise (`docs/ARCHITECTURE.md` "Quantisation policy"; mirrored in
  `quantize.sh`). 49 of the codec's 55 3-D conv tensors are **F16** with non-alignable K
  (7, 3, 2, 1, 16, 10, …) — they cannot be block-quantized at all.
- **No flow matching, no diffusion, no vocoder-with-ODE.** It is a pure RVQ codec
  (Mimi/SoundStream/Mimi-style: SEANet + transformer + DAC decoder). That matters for the
  NPU question: every component is convolution, attention, or embedding lookup.

### 1.3 Where the payload came from, and what the official weights look like

Official upstream (`huggingface.co/api/models`, retrieved 2026-10-19):

| repo | last modified | license | weights |
|---|---|---|---|
| `Qwen/Qwen3-TTS-12Hz-1.7B-Base` | 2026-01-23 | apache-2.0 | `model.safetensors` BF16, 1,928,677,440 params, plus an embedded `speech_tokenizer/model.safetensors` |
| `Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice` | 2026-01-29 | apache-2.0 | same layout, named-speaker tables |
| `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | 2026-01-29 | apache-2.0 | 1.7B only |
| `Qwen/Qwen3-TTS-12Hz-0.6B-Base` / `-0.6B-CustomVoice` | 2026-01-29 | apache-2.0 | narrow variant (hidden 1024) |
| `Qwen/Qwen3-TTS-Tokenizer-12Hz` | 2026-01-29 | apache-2.0 | codec alone, F32, 170,557,441 params |

So the GGUF is **not** the only public form: the safetensors are the canonical form
(Apache-2.0, no separate community license risk), and the GGUF is a lossy re-quantization of
them produced by the same author who wrote the runtime (`Serveurperso`), via
`checkpoints.sh` → `convert.py` (F32 GGUF) → `quantize.sh` (BF16/Q8_0/Q4_K_M).
Sizes to remember for any re-export: the talker container is 1.93 B params in BF16
(that count includes the embedded codec), the codec alone is 170.6 M params in F32.

**Portability consequence #1 — the GGUF is a dead end off-ggml.** Q4_K/Q6_K are ggml
block formats (256-element super-blocks, F16 sub-block scales, 144 B / 210 B per block). No
ONNX graph, no Vitis-AI/AIE-ML compiler, and no vendor NPU runtime consumes them; AMD's own
quantizer works from ONNX (`ryzenai-quantize`, QDQ/QOP power-of-two — Ryzen AI Software
1.8.0 release notes, https://ryzenai.docs.amd.com/en/latest/relnotes.html). Moving the
payload to a non-ggml stack therefore means starting from the safetensors again, not
converting the GGUF.

**Portability consequence #2 — the quantization split is semantic, not cosmetic.** The
converter keeps RVQ codebooks, `input_proj`/`output_proj` around them, and every 1-D and
non-alignable-conv tensor out of the k-quants precisely because quantizing them breaks voice
fidelity. Any re-quantization path (GPTQ/AWQ/QDQ-INT8/INT4 for an NPU compiler) inherits that
constraint: the talker backbone is requantizable, the codec's codebook lookups and SnakeBeta
convs are not, at least not without re-measuring voice quality.

## 2. What the runtime is

`qwentts.cpp` @ `5916bf7c` (2026-10-16, MIT; upstream model Apache-2.0) is a C++17 ggml
application — 43 files under `src/` + `tools/`, three binaries (`qwen-tts`, `qwen-codec`,
`tts-server`) over one public plain-C99 ABI (`src/qwen.h`, `qt_*`, `QT_ABI_VERSION`
range-checked; `tests/abi-c.c` is compiled `-std=c99 -Wall -Werror -pedantic` on every
build). It is not a llama.cpp derivative: it links ggml directly and builds its own graphs.

Five graph families, and their shapes are the whole portability story:

| graph | batch/time axis | notes |
|---|---|---|
| talker prefill | T = prompt length (hundreds–thousands) | summed text+codec embedding streams, 28-layer GQA decoder, KV cache |
| talker decode | **T = 1 per audio frame** | 12.5 frames/s → 80 ms of audio per step; `codec_head` → 3072 logits |
| code predictor (MTP) | **T = 1, 16 steps per frame** (2 prefill + 14 decode, cache reset every frame) | 5-layer decoder, 15 heads × 2048 |
| speaker encoder | one pass per voice registration | ECAPA-TDNN with mel fused into the graph |
| codec decode | offline: T = 300-frame chunks; streaming: **T = 1 frame** with persistent conv/KV-ring state | the streaming graph is built once with fixed topology so the CUDA graph cache stays in pure replay |

Per generated frame the host therefore runs ~1 + 16 + 1 tiny graph computes for 80 ms of
audio, and `tts-server` serializes synthesis FIFO (`--max-batch` defaults to 1, and the
fleet runs it at 1). This is the archetypal latency-bound, bandwidth-bound,
batch-1 workload; the only big-M matmuls in the whole system are the prefill and the
chunked (buffered) codec decode.

Op contract actually issued to ggml (`src/`, counted): `mul_mat` 77, `cont` 70, `add` 57,
`mul` 40, `rms_norm` 22, `clamp` 15, `rope_ext` 12, `get_rows` 12, `set_rows` 8,
`scale_bias` 6, `swiglu_split` 5, `sqr` 5, `mean` 5, `elu` 4, `col2im_1d` 4, `soft_max` 3,
`conv_1d` 3, `im2col` 2, `cumsum` 2, `sin`, `sigmoid`, `tanh`, `div`, `argsort`, `transpose`,
`permute`, `reshape/view`, plus the CPU-side Philox multinomial sampler.

**No custom ggml op is required today.** The repo doc still says the fork "provides the two
custom ops the codec needs: `GGML_OP_SNAKE` and `GGML_OP_COL2IM_1D`"; that is stale at the
pinned commit. Verified 2026-10-19:

- the submodule `ServeurpersoCom/ggml` @ `cc5df49` is upstream `ggml-org/ggml` master
  (`a05c992`, 2026-10-15) **+2 commits, 0 behind** (GitHub compare API), and
  `include/ggml.h` is byte-identical to upstream master (`diff` empty);
- the 2 commits are `193e339` (2026-06-19) "cpu: add fused snake activation
  (mul -> sin -> sqr -> mul -> add)" — a **CPU subgraph fusion**, not an op (the runtime
  builds snake with `ggml_sin`/`ggml_sqr`/`ggml_mul`/`ggml_add` in `src/dac-decoder-v2.h:222`),
  and `cc5df49` "sycl: fix get_rows on reordered quantized weights";
- `ggml_col2im_1d` is an **upstream ggml API** (same declaration, line 2097, in both trees);
  the Vulkan backend fuses the snake chain through ggml's generic subgraph fuser
  (`ggml-vulkan.cpp:14548 ggml_vk_can_fuse_snake`, `vulkan-shaders/snake.comp`).

Consequence: any stock ggml build (upstream, or llama.cpp's vendored ggml) can execute this
payload; only the CPU snake fusion is fork-specific, and it is a performance nicety.

Backend/device support: every tool takes `--device <name>` (`CUDA0`, `Vulkan0`, `MTL0`,
`CPU…`, default best available), resolved through the ggml backend registry; the deployed
container turns CUDA/HIP/Vulkan/Metal/SYCL/RPC/BLAS all OFF and compiles the native-CPU
backend with OpenMP, with CPU threads = `hardware_concurrency()/2` (`src/backend.h`).
The registry in this ggml knows CPU, CUDA, Metal, SYCL, Vulkan, WebGPU, ZDNN, OpenCL,
**Hexagon**, BLAS, RPC, VirtGPU, **CANN**, ZenDNN, **OpenVINO**, ET — and nothing AMD-NPU
(no `xdna`/`aie`/`vitis`/`ryzen` string anywhere in the ggml tree).

Voice handling (what the fleet depends on): the registry is **process RAM only**; a restart
empties it, which is why the unit has a boot warmer. `POST /v1/audio/voices` accepts either
`{name, ref_text, wav_b64}` (server extracts) or `{name, ref_text, spk_b64, rvq_b64}`
(pre-extracted latents verbatim); `ref_text` present ⇒ ICL clone mode, absent ⇒ x-vector-only
mode. `.spk` = the ECAPA x-vector; `.rvq` = 16 codes/frame packed 11 bits LSB-first
(`src/rvq-file.h`). `wav` and `pcm` `response_format` are different audio at the same seed
(buffered chunked decode vs stateful streaming decode) — the fleet already encodes that rule.

Parity expectations for any port: the upstream harness matches the PyTorch reference per
stage (embed cos 0.999988, all 28 hidden taps ≥ 0.999893, logits 0.999824) but explicitly
reports free-run divergence (`CodesFull exact 4.96 %`, `Audio cos 0.095`) and explains why —
one argmax tie at FP epsilon diverts the AR trajectory. So a migrated endpoint must be
accepted by listening + per-stage cosine, never by waveform diff against today's audio.

---

## 3. The three answers Main asked for

### ANSWER A — Qwen3-TTS GGUF → ONNX: no, and it does not matter

There is **no GGUF→ONNX path** and there will not be a useful one: GGUF is a weight
container for ggml, its Q4_K/Q6_K block layouts have no ONNX consumer, and nothing in the
GGUF carries the *graph* (the talker/predictor/codec structure lives in C++ code, only the
op-level structure exists in ggml at runtime). But the GGUF is not the payload's only form —
official Apache-2.0 safetensors are upstream, and **four independent ports of the full
pipeline already exist**:

| port | form | what it proves |
|---|---|---|
| `xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (HF, updated 2026-03-03, apache-2.0) | FP16+FP32 ONNX, fp16/voice_clone + fp16/voice_design splits: `talker_decode.onnx`, `code_predictor.onnx`, `code_predictor_kv.onnx`, `code_predictor_embed_g0..15.onnx`, `text_embedding.onnx`, `codec_embedding.onnx`, shared `speaker_encoder.onnx`, `speech_tokenizer_encoder.onnx`, `speech_tokenizer_decoder.onnx`, `model_cache.npz` | every component of this architecture exports to ONNX, codec decoder included |
| `zhangluoyang/qwen3-tts-onnxruntime` | export tooling + Python inference for Base/CustomVoice/VoiceDesign | the export is reproducible from the official checkpoints |
| `Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO`, `aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03) | OpenVINO IR, **INT8**: `talker`, `code_predictor`, `codec_embedding`, `text_model`, `openvino_speaker_encoder_model`, `speech_tokenizer/speech_decoder` | the talker and predictor INT8-quantize for an NPU-targeting stack |
| `litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17, apache-2.0) | `qwen3_tts_int8.litertlm` / `_int4.litertlm` (LiteRT-LM `OmniEngine`), legacy split graphs `talker_int4.tflite` (prefill_32/prefill_128/decode signatures, KV 1024, blockwise-32 OCTAV INT4), `mtp_folded_int8.tflite` (16 inner steps × 5 layers folded into one graph, in-graph argmax+gather, GPTQ int8, ~5× faster), `codec_partA` (transformer, fp32) + `codec_partB` (conv upsampler, XNNPACK FORCE_FP16, ~2.5×, "ASR-identical") | an NPU/mobile compiler accepts the whole thing — at the price of graph splitting, static prefill buckets, and folding the per-frame inner loop |

What the ports had to concede, and what any port inherits:

- the generation structure does not fit existing engine loops: LiteRT-LM's Engine "does not
  support this generation structure yet", so their first shape was **three graphs + a
  host-side loop**, and the MTP step was "invoked 17× per frame" until they folded it;
- per-step dispatch cost dominates: even on a Snapdragon 8 Elite Gen 5 they keep the MTP
  loop, text projection, embedding gathers and the codec on CPU "avoiding 16 synchronous
  CPU↔GPU round-trips per 80 ms frame", putting only the 28-layer talker on the GPU;
- RTF on that hardware is **2.57–2.97 (Galaxy S26) and 5.65–8.48 (Pixel 10 Pro)** for the
  0.6 B model, i.e. 2.6–8× slower than real time. Dankclaw's current CPU engine measures
  **warm RTF 0.55–0.67 with TTFA 94–112 ms** on the 1.7 B model (fleet trial, 2026-10-18).
  An NPU port is not "faster by default" for this workload; it has to beat that.
- quantize the LM (int8/int4: proven), keep the codec's codebook lookups and SnakeBeta convs
  in fp16/fp32 (the LiteRT split does exactly this).

So: the answer to "can the fleet's payload be exported" is *yes — from the safetensors, not
from the GGUF*, and the work is a pipeline rewrite (multi-graph host loop, per-frame fold,
static prefill buckets), not a conversion. That is the same conclusion NeuTTS-Air's
precompiled-graph route encodes: on AMD's Linux stack the graph is vendor-curated, so the
fleet chooses the model that AMD already compiled rather than compiling Qwen3-TTS.

### ANSWER B — is there any ggml backend for an AMD NPU? Not upstream; three out-of-tree, none for TTS

Checked against the trees today (2026-10-19), because this was the crux:

- **Nothing in-tree.** `ggml-org/llama.cpp` master (4,237 paths, GitHub tree API) contains no
  `xdna`/`aie`/`vitis`/`ryzen` file or directory; the ggml backend set is CPU, BLAS, CUDA,
  HIP, Metal, Vulkan, SYCL, OpenCL, WebGPU, MUSA, RPC, CANN (Ascend), **Hexagon** (Qualcomm),
  OpenVINO (Intel CPU/GPU/NPU), ZenDNN, ZDNN, VirtGPU, ET. The deployed fork's registry
  matches. The feature request "Feature Request: XDNA backend" (llama.cpp issue **#21725**,
  opened 2026-04-10) is still **open**, 34 comments; the older #1499 was closed stale
  2025-04-07. So Main's premise is confirmed: no ggml XDNA backend exists to flip a flag on.
- **`Cyronius/ggml-xdna`** — a real ggml backend (`XDNA0`) loaded into *stock* llama.cpp via
  `GGML_BACKEND_PATH`, no fork. It does bfp16 (8-value blocks sharing an exponent) GEMMs from
  GGUF weights, and is **prefill-only**: it takes only matmuls in prompt chunks ≥ 1024 tokens
  (`GGML_XDNA_MIN_BATCH` default 1024, `GGML_XDNA_MIN_MFLOP` 256), attention and every decode
  token stay on the Vulkan device; pinned to llama.cpp release b10944; tested on Strix Point
  (HX 370) only — Strix Halo "should pass, but nobody has run it on one yet"; Linux support
  new and untested on hardware by the author; needs `libxrt-npu2`, `/dev/accel/accel0`,
  unlimited `memlock`. Best measured prefill gain 1.88× on a 27 B model; models under
  2,048-wide **lose** (0.77× at width 1,536).
- **`BrandedTamarasu-glitch/OllamaAMDNPU`** — a llama.cpp fork with `-DGGML_XDNA=ON`,
  explicitly targeting **Ryzen AI MAX / XDNA2 on Linux** (`/dev/accel/accel0`, XRT 2.21.75+,
  amdxdna, unlimited memlock). It implements **only `GGML_OP_MUL_MAT`**, dispatched to
  mlir-aie-compiled `.xclbin`s built per fixed tile shape (e.g. `M=2048 K=2048 N=64`,
  int8 in / int32 out, per-row weight scales cached at first use), with a second slot for a
  different K. Its own numbers on a Ryzen AI MAX 385 with Llama-3.1-8B Q4_K_M: NPU prefill
  12–19 t/s vs **Vulkan iGPU 600–930 t/s**, NPU decode **0.65–1.40 t/s** ("Decode does not
  use the NPU … M=1 per token, no xclbin covers it"), 0.6 % AIE utilization, and the stated
  conclusion "**For LLaMA 8B on Strix Halo: use Vulkan exclusively**". Single-maintainer,
  AI-assisted, self-measured — treat as a proof of concept, but it is the closest published
  picture of what XDNA2 graph execution costs.
- Also referenced in the same thread: `Atomic-Germ/OpenFlowLM-Next` (whole-model NPU engine,
  LLM-only — and slower than the iGPU in the same comparison), `GaTmaNnes/xdna2-`,
  `hardWorker254/Npu-Embeddings-XDNA1` (adds Whisper ASR ops, NPU1/Linux). None is a TTS path.

Why none of them would carry this workload even if installed: the TTS hot loop is **batch-1
by construction** (one talker step + 16 predictor steps + one codec frame per 80 ms), which
sits under every dispatch gate those backends define (≥1024 tokens / ≥32768 M·N·K / tile M
covering only prefill), and the codec half of the pipeline is conv, col2im, gather and
elementwise ops — not `MUL_MAT` — so it would fall back to CPU op-by-op. The NPU case for
this host is therefore *not* "run Qwen3-TTS on the NPU"; it is "run a vendor-compiled AR TTS
graph on the NPU", which is exactly the NeuTTS-Air spike.

### ANSWER C — would `qwentts.cpp` build for the gfx1151 iGPU? Yes, via **Vulkan**, not HIP

- Upstream ships the build scripts and claims the backends: `buildvulkan.sh` ("AMD/Intel GPU
  (Vulkan)"), `buildcuda.sh`, `buildsycl.sh`, `buildcpu.sh`, `buildall.sh` (all backends,
  runtime dynamic loading), and README line 6 "runs on CPU, CUDA, Metal, Vulkan". CMake knobs
  `GGML_VULKAN` / `GGML_HIP` / `GGML_SYCL` are the same ones the fleet's Containerfile turns
  OFF, so the engine rebuild is a flag change: `-DGGML_VULKAN=ON` plus
  `--device Vulkan0` on the same command line the unit runs today.
- The codec's exotic ops are covered by the Vulkan backend, which is the thing that decides
  whether a GPU build is coherent: `vulkan-shaders/col2im_1d.comp`, `snake.comp`,
  `swiglu*.comp`, `cumsum*.comp` exist and `supports_op` handles `GGML_OP_COL2IM_1D` /
  `CONV_TRANSPOSE_1D` / `CUMSUM`. By contrast `COL2IM_1D` appears in **no** HIP, OpenCL,
  WebGPU, Hexagon or CANN source — a **HIP** (ROCm) build would shard the ConvNeXt/DAC
  transposed convs back to the CPU per op. On an AMD iGPU the portable path is RADV/Vulkan.
- `--device` also accepts a multi-device split (`-dev`-style naming is ggml's), and the
  runtime's documented FP16 guard (`--clamp-fp16`) exists for sub-Ampere CUDA accumulation;
  nothing in the tree is NVIDIA-specific.
- Unverified by me: any vendor statement pairing this fork with gfx1151. Supporting evidence
  that the hardware/driver combination is viable: this very host already runs a Vulkan
  endpoint on the gfx1151 iGPU (fleet ground truth, 2026-10-19), and Mesa's Vulkan
  implementation is officially supported on Radeon Software for Linux
  (AMD release notes RN-AMDGPU-UNIFIED-LINUX-25.20.3). A gfx1151 Vulkan build of this fork is
  a build-and-measure spike, not an unsupported port.
- The real constraint is the one already in the fleet docs: the iGPU belongs to another
  endpoint, and GPU+NPU concurrency has hung Strix Halo machines. An iGPU TTS build competes
  with the text endpoints for the same silicon; the NPU route's only genuine advantage is
  that it does not (the XDNA backend authors make the same argument: "Use NPU when the iGPU
  is busy … no GPU contention").

---

## 4. What any replacement must reproduce, and what the voices cost

Endpoint contract in production today (from the live unit/env, not the doc):
`tts-server --model qwen-talker-1.7b-base-Q4_K_M.gguf --codec qwen-tokenizer-12hz-Q8_0.gguf
--alias frank-tts-engine --host 0.0.0.0 --port 8880 --lang auto --max-batch 1`, no published
port (the façade on :7772 dials the private bridge name), `GET /health`, `GET /v1/models`,
`GET|POST /v1/audio/voices`, `DELETE /v1/audio/voices/{name}`, `POST /v1/audio/speech` with
`response_format` wav|pcm (different audio at equal seed) and optional `seed`,
`max_new_tokens`, `temperature`, `top_k`, `top_p`, `repetition_penalty`, `subtalker_*`.
Voice registry is RAM-only, so boot warming is mandatory, and `--alias` keeps the quantization
choice out of every consumer's config.

Live voice census on Dankclaw (2026-10-19, `/home/dank/voices/`): three packs
(`narrator1..3`), each carrying `reference.wav` (24 kHz mono crop), `reference16k.wav`,
`manifest.json` (keys: name, kind, generated_utc, pipeline, source, forbidden_region_guard,
silence_scan, crop, loudnorm, audio, acceptance, acceptance_passed) and ffmpeg analysis
artifacts. **No `.spk` and no `.rvq` exist on the live host** — the repo's
`build-voice-pack.sh` treats them as required-but-they-are-not-there, so today the engine
re-extracts the x-vector (and the ICL codes when a transcript is supplied) from the WAV at
registration time.

That is the portable part, and it is the same shape AMD's NeuTTS flow wants
(`encode_reference(reference.wav)` + a reference transcript): the fleet's voice identity is
*the WAV plus its transcript*, processed at request/registration time by the model's own
speaker encoder + codec — **never a trained per-voice artifact**. Qwen3-TTS cloning is
inference-time conditioning (`generate_voice_clone(ref_audio=…, ref_text=…, x_vector_only_mode=…)`;
`create_voice_clone_prompt` exists purely to avoid recomputing prompt features). So any
replacement that consumes a WAV (+transcript) keeps all three narrators alive with no
re-derivation, and if it wants precomputed latents instead, the LiteRT precedent shows what
that looks like: a 4 KB x-vector `.npy` per voice, extracted offline
(`extract_speaker_embedding.py`) — for the 1.7 B base checkpoint that vector is 2048-dim
(`qwen3-tts.spk_enc.embedding_length = 2048` in the shipped GGUF; the LiteRT 0.6 B bundle
ships a 1024-dim one).

One trap for a mixed-runtime future: `.spk`/`.rvq` files, if they are ever written, are
engine-specific encodings (ECAPA x-vector layout; 11-bit-packed RVQ stream) and are **not**
interchangeable between runtimes. Keep `reference.wav` as the pack's source of truth and
regenerate anything derived per engine — which is what the NeuTTS re-clone plan does anyway.

## 5. Sources

Runtime/payload (all read locally at pinned versions, 2026-10-19):

- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
  (commit date 2026-10-16; also upstream HEAD today): `README.md`, `docs/ARCHITECTURE.md`
  (860 lines), `src/*`, `tools/*`, `docs/DOCKER.md`, `buildvulkan.sh`/`buildcpu.sh`/`buildall.sh`.
- submodule `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16);
  delta vs `ggml-org/ggml` master `a05c992` (2026-10-15) = ahead 2 / behind 0, per
  https://github.com/ServeurpersoCom/ggml/compare/ggml-org:master...HEAD ;
  `include/ggml.h` identical to upstream
  https://raw.githubusercontent.com/ggml-org/ggml/master/include/ggml.h .
- Live payload bytes: `/home/dank/models/Qwen3-TTS-GGUF/qwen-talker-1.7b-base-Q4_K_M.gguf`
  (1,219,245,248 B) and `qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B), headers parsed from
  the host; sizes/sha256 match `scripts/download-frank-tts-engine-qwentts.sh`.
- Live units/env/packs: `dankclaw/voiceservice/{Containerfile.engine,tts-engine-start.sh,tts-engine-warm.sh,quadlet/*}`,
  `scripts/frank-tts-engine.env`, `scripts/frank-tts.env`, `/home/dank/voices/*`.

Model provenance:

- `https://huggingface.co/api/models/Qwen/Qwen3-TTS-12Hz-1.7B-Base` (+ CustomVoice,
  VoiceDesign, 0.6B variants, `Qwen3-TTS-Tokenizer-12Hz`), apache-2.0, last-modified
  2026-01-23 / 2026-01-29; safetensors param counts 1,928,677,440 (BF16, incl. embedded
  `speech_tokenizer/`) and 170,557,441 (F32 codec).
- `https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base/raw/main/README.md` (clone API:
  `generate_voice_clone(ref_audio, ref_text, x_vector_only_mode)`,
  `create_voice_clone_prompt`; reference stack = PyTorch + `device_map="cuda:0"` + bf16 +
  flash-attn-2, Python 3.12 recommended).
- `https://github.com/QwenLM/Qwen3-TTS`.

Portability precedents:

- `https://huggingface.co/xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (2026-03-03) ·
  `https://github.com/zhangluoyang/qwen3-tts-onnxruntime` ·
  `https://huggingface.co/Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO` (2026-03-22) ·
  `https://huggingface.co/aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03-18) ·
  `https://huggingface.co/litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17) incl.
  `legacy/README.md` (graph splits, MTP fold, codec partA/partB, RTF table).

AMD NPU / ggml backend status:

- `https://api.github.com/repos/ggml-org/llama.cpp/git/trees/master?recursive=1` (no
  XDNA/AIE/Vitis backend, 2026-10-19) ·
  `https://github.com/ggml-org/llama.cpp/issues/21725` (open since 2026-04-10) ·
  `https://github.com/ggml-org/llama.cpp/issues/1499` (closed stale 2025-04-07).
- `https://github.com/Cyronius/ggml-xdna` (prefill-only, gates, Linux requirements, results
  measured 2026-10-05/06 on HX 370) ·
  `https://github.com/BrandedTamarasu-glitch/OllamaAMDNPU` (XDNA2 ggml backend on Strix Halo,
  phases 1–12, "use Vulkan exclusively", measured on Ryzen AI MAX 385).
- `https://github.com/amd/RyzenAI-SW` (main, 669 paths, 2026-08-18): audio content is ASR
  only (`Demos/ASR/Whisper`, `Demos/ASR/Parakeet-TDT`, `Transformer-examples/ASR/Whisper-AI`);
  **no TTS example anywhere in the official Ryzen AI Software tree**.
- `https://ryzenai.docs.amd.com/en/latest/relnotes.html` (Ryzen AI SW 1.8.0: ONNX-format
  models, ONNX Runtime as the loader, power-of-two QDQ/QOP quantization).
- Mesa/RADV on AMD iGPUs: `https://www.amd.com/en/resources/support-articles/release-notes/RN-AMDGPU-UNIFIED-LINUX-25-20.3.html`
  (Mesa Vulkan driver officially supported).

Status: COMPLETE for the assigned slice (payload architecture, runtime architecture,
portability). `[INFERENCE]` marks: none above except the Mesa-version caveat in ANSWER C,
which is flagged there.