serverAppConfigs/report/qwen3-tts-payload-portability-2026-10-19.md

Qwen3-TTS payload + qwentts.cpp runtime: architecture and portability

Research deliverable (no code). Slice: what the fleet's TTS payload actually is, and how portable it is off the CPU ggml path. Companion slice: NpuRuntimeLandscape (what NPU runtimes exist on Dankclaw).

Evidence date: 2026-10-19. Everything below is grounded in:

• github.com/ServeurpersoCom/qwentts.cpp @ 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4 (2026-10-16, upstream HEAD on 2026-10-19) and its ggml submodule ServeurpersoCom/ggml @ cc5df49a6b878db2b68b9a7b45976e279594627d (2026-10-16).
• The two GGUF files as they sit on Dankclaw (/home/dank/models/Qwen3-TTS-GGUF/), read byte-for-byte from the live host.
• Upstream Qwen3-TTS model cards / release material.

Status: COMPLETE for the assigned slice. ANSWER C was confirmed on the live host the same night — the running frank-tts-engine container carries libggml-base.so.0.27.0 + libggml-cpu.so.0.27.0 and nothing else, and our own dankclaw/voiceservice/Containerfile.engine writes -DGGML_VULKAN=OFF out explicitly, so the iGPU change is a flag in a file the fleet owns. Verdict, measured contract and the NPU exoneration chain: tts-npu-spike-verdict-2026-10-19.md.

---

1. What the payload is

Two files, both read byte-for-byte off the live host (/home/dank/models/Qwen3-TTS-GGUF/, owned dank:dank, mtimes 2026-10-18 18:31), both from https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF (community converter, not Qwen):

file	bytes	GGUF	tensors	arch string	file_type
qwen-talker-1.7b-base-Q4_K_M.gguf	1,219,245,248	v3, kv=64	480	qwen3-tts	Q4_K_M
qwen-tokenizer-12hz-Q8_0.gguf	291,150,624	v3, kv=51	398	qwen3-tts-tokenizer	Q8_0

general.name of the talker file is Qwen3-TTS-12Hz-1.7B-base; the header sizes add up to the file sizes (sum of tensor bytes + header + alignment padding), so nothing in these files is unaccounted for.

1.1 Talker file (480 tensors)

Three models in one container, per docs/ARCHITECTURE.md §GGUF layout (qwentts.cpp @ pinned commit) and confirmed by the tensor census:

prefix	tensors	bytes	what it is
talker.*	316	1,061.9 MB	28-layer Qwen3 decoder (hidden 2048, FFN 6144, 16 Q heads / 8 KV heads, head_dim 128, RoPE θ=1e6, RMS ε=1e-6, ctx 32768), text stream + codec stream summed, codec_head → 3072 codebook-0 logits
code_pred.*	88	114.8 MB	5-layer Qwen3 MTP head (hidden 1024, FFN 3072, 15 acoustic codebooks × 2048), own KV cache
spk_enc.*	76	36.6 MB	ECAPA-TDNN speaker encoder (x-vector, base checkpoints only; mel → conv0 → 3 SE-Res2Net → MFA → ASP → 2048-d vector)

Dtype split (exact tensor byte sums): q4_K 250 tensors / 864.1 MB, q6_K 17 / 312.0 MB, f32 176 / 25.8 MB, f16 37 / 11.4 MB.

Which tensors carry which quant is the load-bearing portability fact:

• q6_K (17): talker.text_embd.weight [2048, 151936] (255.3 MB), talker.codec_embd.weight [2048, 3072], and the 15 code_pred.codec_embd.<g>.weight [2048, 2048] embedding tables. These are the llama.cpp Q4_K_M "embedding/output rows go to Q6_K" rule, applied to embedding tables rather than to an lm_head.
• q4_K (250): every attention/FFN projection of the talker and of the code predictor. Distinct matmul shapes: [2048,2048]×73, [2048,1024]×62, [2048,6144]×56, [6144,2048]×28, [1024,2048]×20, [1024,3072]×10, [1024,1024]×10, [3072,1024]×5, [2048,3072]×2, [2048,151936]×1. Every one is 2-D and ≥ 1024 on both axes — good GEMM shapes, nothing exotic.
• f32 (176) / f16 (37): all of spk_enc (conv weights F16 because K=1/3/5 rows are not block-alignable, per the converter's should_quantize policy in tools/quantize.cpp + quantize.sh; every bias/norm/LayerScale vector F32), plus the talker/predictor norms and the text_proj biases.

Metadata the runtime actually reads (talker file): qwen3-tts.num_code_groups=16, talker.vocab_size=3072, text_vocab_size=151936, text_hidden_size=2048, position_id_per_seconds=13, rope.mrope_section=[24,20,20], mrope_interleaved=1, the 7 codec special ids (pad 2148 / bos 2149 / eos 2150 / think 2154 / nothink 2155 / think_bos 2156 / think_eos 2157), language_names (10: chinese, english, german, italian, portuguese, spanish, japanese, korean, french, russian) with their codec ids, the text-side special ids (im_start 151644, im_end 151645, tts_pad 151671, tts_bos 151672, tts_eos 151673), the whole generation_config.json (do_sample, top_k 50, top_p 1.0, temperature 0.9, repetition_penalty 1.05, subtalker_* mirrors, max_new_tokens 8192), and an embedded Qwen2 BPE tokenizer (tokenizer.ggml.model=gpt2, 151,676 tokens, 151,291 merges, eos 151643) — that BPE block is 5.95 MB of the header, i.e. 49 % of the header is vocabulary.

Two live-file details that contradict the upstream doc text (worth knowing when reading docs/ARCHITECTURE.md: the doc example block is stale against the shipped file): mrope_interleaved is 1 (doc shows false), and spk_enc.sample_rate is 24000 (doc says 16000). Doc also says "404 tensors on the 1.7B voice_design build" — the base build shipped here has 480 because it carries spk_enc.*.

1.2 Tokenizer/codec file (398 tensors)

One neural audio codec, 12.5 frames/s over 24 kHz mono, hop 1920, 16 codebooks × 2048 entries (1 semantic + 15 acoustic). Dtype split: f16 50 / 143.6 MB, q8_0 110 / 76.9 MB, f32 238 / 70.7 MB. tok_dec.* 253 tensors / 201.3 MB, tok_enc.* 145 / 89.9 MB.

Composition (metadata + tensor census, both files agree):

• Encoder (used only to make voice references, not on the synthesis hot path): SEANet conv stack (init k=7, 4 stages with ratios 8/6/5/4 → cumulative 960×, 64→512 ch, final k=3), an 8-layer transformer (hidden 512, 8 heads, head_dim 64, FFN 2048, RoPE 1e4, LayerNorm+bias, GELU, LayerScale), a downsample conv k=4 s=2 (25 → 12.5 Hz), then the RVQ argmin loop. Metadata records encoder.num_quantizers=32 with encoder_valid_num_quantizers=16 — 16 codebooks are used, 16 are dead weight carried from the upstream config.
• Decoder (the hot path: every generated frame): 16 codebook lookups + per-split output_proj (256→512), conv_pre, an 8-layer transformer (hidden 512, 16/16 heads, head_dim 64, FFN 1024, sliding window 72, RMSNorm, SwiGLU, LayerScale, input_proj 1024→512 / output_proj 512→1024), a ConvNeXt upampler (upsampling_ratios=[2,2], causal transposed convs + depthwise k=7), then a DAC-style decoder dec.0..3 (upsample_rates=[8,5,4,3], channels 1536→768→384→192→96, decoder_dim=1536) whose blocks are SnakeBeta → causal transposed conv (k = 2·stride) → 3 residual units (dilations 1/3/9) → snake_post → conv_post k=7 → 96→1.
• All 32 codebook tensors are F32, 67.1 MB total (16 encode + 16 decode, [256, 2048]), deliberately unquantized: the converter's should_quantize keeps RVQ codebooks and their wrapping projections at F32 because nearest-neighbour lookup is sensitive to per-row quantisation noise (docs/ARCHITECTURE.md "Quantisation policy"; mirrored in quantize.sh). 49 of the codec's 55 3-D conv tensors are F16 with non-alignable K (7, 3, 2, 1, 16, 10, …) — they cannot be block-quantized at all.
• No flow matching, no diffusion, no vocoder-with-ODE. It is a pure RVQ codec (Mimi/SoundStream/Mimi-style: SEANet + transformer + DAC decoder). That matters for the NPU question: every component is convolution, attention, or embedding lookup.

1.3 Where the payload came from, and what the official weights look like

Official upstream (huggingface.co/api/models, retrieved 2026-10-19):

repo	last modified	license	weights
Qwen/Qwen3-TTS-12Hz-1.7B-Base	2026-01-23	apache-2.0	model.safetensors BF16, 1,928,677,440 params, plus an embedded speech_tokenizer/model.safetensors
Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice	2026-01-29	apache-2.0	same layout, named-speaker tables
Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign	2026-01-29	apache-2.0	1.7B only
Qwen/Qwen3-TTS-12Hz-0.6B-Base / -0.6B-CustomVoice	2026-01-29	apache-2.0	narrow variant (hidden 1024)
Qwen/Qwen3-TTS-Tokenizer-12Hz	2026-01-29	apache-2.0	codec alone, F32, 170,557,441 params

So the GGUF is not the only public form: the safetensors are the canonical form (Apache-2.0, no separate community license risk), and the GGUF is a lossy re-quantization of them produced by the same author who wrote the runtime (Serveurperso), via checkpoints.sh → convert.py (F32 GGUF) → quantize.sh (BF16/Q8_0/Q4_K_M). Sizes to remember for any re-export: the talker container is 1.93 B params in BF16 (that count includes the embedded codec), the codec alone is 170.6 M params in F32.

Portability consequence #1 — the GGUF is a dead end off-ggml. Q4_K/Q6_K are ggml block formats (256-element super-blocks, F16 sub-block scales, 144 B / 210 B per block). No ONNX graph, no Vitis-AI/AIE-ML compiler, and no vendor NPU runtime consumes them; AMD's own quantizer works from ONNX (ryzenai-quantize, QDQ/QOP power-of-two — Ryzen AI Software 1.8.0 release notes, https://ryzenai.docs.amd.com/en/latest/relnotes.html). Moving the payload to a non-ggml stack therefore means starting from the safetensors again, not converting the GGUF.

Portability consequence #2 — the quantization split is semantic, not cosmetic. The converter keeps RVQ codebooks, input_proj/output_proj around them, and every 1-D and non-alignable-conv tensor out of the k-quants precisely because quantizing them breaks voice fidelity. Any re-quantization path (GPTQ/AWQ/QDQ-INT8/INT4 for an NPU compiler) inherits that constraint: the talker backbone is requantizable, the codec's codebook lookups and SnakeBeta convs are not, at least not without re-measuring voice quality.

2. What the runtime is

qwentts.cpp @ 5916bf7c (2026-10-16, MIT; upstream model Apache-2.0) is a C++17 ggml application — 43 files under src/ + tools/, three binaries (qwen-tts, qwen-codec, tts-server) over one public plain-C99 ABI (src/qwen.h, qt_*, QT_ABI_VERSION range-checked; tests/abi-c.c is compiled -std=c99 -Wall -Werror -pedantic on every build). It is not a llama.cpp derivative: it links ggml directly and builds its own graphs.

Five graph families, and their shapes are the whole portability story:

graph	batch/time axis	notes
talker prefill	T = prompt length (hundreds–thousands)	summed text+codec embedding streams, 28-layer GQA decoder, KV cache
talker decode	T = 1 per audio frame	12.5 frames/s → 80 ms of audio per step; codec_head → 3072 logits
code predictor (MTP)	T = 1, 16 steps per frame (2 prefill + 14 decode, cache reset every frame)	5-layer decoder, 15 heads × 2048
speaker encoder	one pass per voice registration	ECAPA-TDNN with mel fused into the graph
codec decode	offline: T = 300-frame chunks; streaming: T = 1 frame with persistent conv/KV-ring state	the streaming graph is built once with fixed topology so the CUDA graph cache stays in pure replay

Per generated frame the host therefore runs ~1 + 16 + 1 tiny graph computes for 80 ms of audio, and tts-server serializes synthesis FIFO (--max-batch defaults to 1, and the fleet runs it at 1). This is the archetypal latency-bound, bandwidth-bound, batch-1 workload; the only big-M matmuls in the whole system are the prefill and the chunked (buffered) codec decode.

Op contract actually issued to ggml (src/, counted): mul_mat 77, cont 70, add 57, mul 40, rms_norm 22, clamp 15, rope_ext 12, get_rows 12, set_rows 8, scale_bias 6, swiglu_split 5, sqr 5, mean 5, elu 4, col2im_1d 4, soft_max 3, conv_1d 3, im2col 2, cumsum 2, sin, sigmoid, tanh, div, argsort, transpose, permute, reshape/view, plus the CPU-side Philox multinomial sampler.

No custom ggml op is required today. The repo doc still says the fork "provides the two custom ops the codec needs: GGML_OP_SNAKE and GGML_OP_COL2IM_1D"; that is stale at the pinned commit. Verified 2026-10-19:

• the submodule ServeurpersoCom/ggml @ cc5df49 is upstream ggml-org/ggml master (a05c992, 2026-10-15) +2 commits, 0 behind (GitHub compare API), and include/ggml.h is byte-identical to upstream master (diff empty);
• the 2 commits are 193e339 (2026-06-19) "cpu: add fused snake activation (mul -> sin -> sqr -> mul -> add)" — a CPU subgraph fusion, not an op (the runtime builds snake with ggml_sin/ggml_sqr/ggml_mul/ggml_add in src/dac-decoder-v2.h:222), and cc5df49 "sycl: fix get_rows on reordered quantized weights";
• ggml_col2im_1d is an upstream ggml API (same declaration, line 2097, in both trees); the Vulkan backend fuses the snake chain through ggml's generic subgraph fuser (ggml-vulkan.cpp:14548 ggml_vk_can_fuse_snake, vulkan-shaders/snake.comp).

Consequence: any stock ggml build (upstream, or llama.cpp's vendored ggml) can execute this payload; only the CPU snake fusion is fork-specific, and it is a performance nicety.

Backend/device support: every tool takes --device <name> (CUDA0, Vulkan0, MTL0, CPU…, default best available), resolved through the ggml backend registry; the deployed container turns CUDA/HIP/Vulkan/Metal/SYCL/RPC/BLAS all OFF and compiles the native-CPU backend with OpenMP, with CPU threads = hardware_concurrency()/2 (src/backend.h). The registry in this ggml knows CPU, CUDA, Metal, SYCL, Vulkan, WebGPU, ZDNN, OpenCL, Hexagon, BLAS, RPC, VirtGPU, CANN, ZenDNN, OpenVINO, ET — and nothing AMD-NPU (no xdna/aie/vitis/ryzen string anywhere in the ggml tree).

Voice handling (what the fleet depends on): the registry is process RAM only; a restart empties it, which is why the unit has a boot warmer. POST /v1/audio/voices accepts either {name, ref_text, wav_b64} (server extracts) or {name, ref_text, spk_b64, rvq_b64} (pre-extracted latents verbatim); ref_text present ⇒ ICL clone mode, absent ⇒ x-vector-only mode. .spk = the ECAPA x-vector; .rvq = 16 codes/frame packed 11 bits LSB-first (src/rvq-file.h). wav and pcm response_format are different audio at the same seed (buffered chunked decode vs stateful streaming decode) — the fleet already encodes that rule.

Parity expectations for any port: the upstream harness matches the PyTorch reference per stage (embed cos 0.999988, all 28 hidden taps ≥ 0.999893, logits 0.999824) but explicitly reports free-run divergence (CodesFull exact 4.96 %, Audio cos 0.095) and explains why — one argmax tie at FP epsilon diverts the AR trajectory. So a migrated endpoint must be accepted by listening + per-stage cosine, never by waveform diff against today's audio.

---

3. The three answers Main asked for

ANSWER A — Qwen3-TTS GGUF → ONNX: no, and it does not matter

There is no GGUF→ONNX path and there will not be a useful one: GGUF is a weight container for ggml, its Q4_K/Q6_K block layouts have no ONNX consumer, and nothing in the GGUF carries the graph (the talker/predictor/codec structure lives in C++ code, only the op-level structure exists in ggml at runtime). But the GGUF is not the payload's only form — official Apache-2.0 safetensors are upstream, and four independent ports of the full pipeline already exist:

port	form	what it proves
xkos/Qwen3-TTS-12Hz-1.7B-ONNX (HF, updated 2026-03-03, apache-2.0)	FP16+FP32 ONNX, fp16/voice_clone + fp16/voice_design splits: talker_decode.onnx, code_predictor.onnx, code_predictor_kv.onnx, code_predictor_embed_g0..15.onnx, text_embedding.onnx, codec_embedding.onnx, shared speaker_encoder.onnx, speech_tokenizer_encoder.onnx, speech_tokenizer_decoder.onnx, model_cache.npz	every component of this architecture exports to ONNX, codec decoder included
zhangluoyang/qwen3-tts-onnxruntime	export tooling + Python inference for Base/CustomVoice/VoiceDesign	the export is reproducible from the official checkpoints
Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO, aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8 (2026-03)	OpenVINO IR, INT8: talker, code_predictor, codec_embedding, text_model, openvino_speaker_encoder_model, speech_tokenizer/speech_decoder	the talker and predictor INT8-quantize for an NPU-targeting stack
litert-community/Qwen3-TTS-12Hz-0.6B-Base (2026-10-17, apache-2.0)	qwen3_tts_int8.litertlm / _int4.litertlm (LiteRT-LM OmniEngine), legacy split graphs talker_int4.tflite (prefill_32/prefill_128/decode signatures, KV 1024, blockwise-32 OCTAV INT4), mtp_folded_int8.tflite (16 inner steps × 5 layers folded into one graph, in-graph argmax+gather, GPTQ int8, ~5× faster), codec_partA (transformer, fp32) + codec_partB (conv upsampler, XNNPACK FORCE_FP16, ~2.5×, "ASR-identical")	an NPU/mobile compiler accepts the whole thing — at the price of graph splitting, static prefill buckets, and folding the per-frame inner loop

What the ports had to concede, and what any port inherits:

• the generation structure does not fit existing engine loops: LiteRT-LM's Engine "does not support this generation structure yet", so their first shape was three graphs + a host-side loop, and the MTP step was "invoked 17× per frame" until they folded it;
• per-step dispatch cost dominates: even on a Snapdragon 8 Elite Gen 5 they keep the MTP loop, text projection, embedding gathers and the codec on CPU "avoiding 16 synchronous CPU↔GPU round-trips per 80 ms frame", putting only the 28-layer talker on the GPU;
• RTF on that hardware is 2.57–2.97 (Galaxy S26) and 5.65–8.48 (Pixel 10 Pro) for the 0.6 B model, i.e. 2.6–8× slower than real time. Dankclaw's current CPU engine measures warm RTF 0.55–0.67 with TTFA 94–112 ms on the 1.7 B model (fleet trial, 2026-10-18). An NPU port is not "faster by default" for this workload; it has to beat that.
• quantize the LM (int8/int4: proven), keep the codec's codebook lookups and SnakeBeta convs in fp16/fp32 (the LiteRT split does exactly this).

So: the answer to "can the fleet's payload be exported" is yes — from the safetensors, not from the GGUF, and the work is a pipeline rewrite (multi-graph host loop, per-frame fold, static prefill buckets), not a conversion. That is the same conclusion NeuTTS-Air's precompiled-graph route encodes: on AMD's Linux stack the graph is vendor-curated, so the fleet chooses the model that AMD already compiled rather than compiling Qwen3-TTS.

ANSWER B — is there any ggml backend for an AMD NPU? Not upstream; three out-of-tree, none for TTS

Checked against the trees today (2026-10-19), because this was the crux:

• Nothing in-tree. ggml-org/llama.cpp master (4,237 paths, GitHub tree API) contains no xdna/aie/vitis/ryzen file or directory; the ggml backend set is CPU, BLAS, CUDA, HIP, Metal, Vulkan, SYCL, OpenCL, WebGPU, MUSA, RPC, CANN (Ascend), Hexagon (Qualcomm), OpenVINO (Intel CPU/GPU/NPU), ZenDNN, ZDNN, VirtGPU, ET. The deployed fork's registry matches. The feature request "Feature Request: XDNA backend" (llama.cpp issue #21725, opened 2026-04-10) is still open, 34 comments; the older #1499 was closed stale 2025-04-07. So Main's premise is confirmed: no ggml XDNA backend exists to flip a flag on.
• Cyronius/ggml-xdna — a real ggml backend (XDNA0) loaded into stock llama.cpp via GGML_BACKEND_PATH, no fork. It does bfp16 (8-value blocks sharing an exponent) GEMMs from GGUF weights, and is prefill-only: it takes only matmuls in prompt chunks ≥ 1024 tokens (GGML_XDNA_MIN_BATCH default 1024, GGML_XDNA_MIN_MFLOP 256), attention and every decode token stay on the Vulkan device; pinned to llama.cpp release b10944; tested on Strix Point (HX 370) only — Strix Halo "should pass, but nobody has run it on one yet"; Linux support new and untested on hardware by the author; needs libxrt-npu2, /dev/accel/accel0, unlimited memlock. Best measured prefill gain 1.88× on a 27 B model; models under 2,048-wide lose (0.77× at width 1,536).
• BrandedTamarasu-glitch/OllamaAMDNPU — a llama.cpp fork with -DGGML_XDNA=ON, explicitly targeting Ryzen AI MAX / XDNA2 on Linux (/dev/accel/accel0, XRT 2.21.75+, amdxdna, unlimited memlock). It implements only GGML_OP_MUL_MAT, dispatched to mlir-aie-compiled .xclbins built per fixed tile shape (e.g. M=2048 K=2048 N=64, int8 in / int32 out, per-row weight scales cached at first use), with a second slot for a different K. Its own numbers on a Ryzen AI MAX 385 with Llama-3.1-8B Q4_K_M: NPU prefill 12–19 t/s vs Vulkan iGPU 600–930 t/s, NPU decode 0.65–1.40 t/s ("Decode does not use the NPU … M=1 per token, no xclbin covers it"), 0.6 % AIE utilization, and the stated conclusion "For LLaMA 8B on Strix Halo: use Vulkan exclusively". Single-maintainer, AI-assisted, self-measured — treat as a proof of concept, but it is the closest published picture of what XDNA2 graph execution costs.
• Also referenced in the same thread: Atomic-Germ/OpenFlowLM-Next (whole-model NPU engine, LLM-only — and slower than the iGPU in the same comparison), GaTmaNnes/xdna2-, hardWorker254/Npu-Embeddings-XDNA1 (adds Whisper ASR ops, NPU1/Linux). None is a TTS path.

Why none of them would carry this workload even if installed: the TTS hot loop is batch-1 by construction (one talker step + 16 predictor steps + one codec frame per 80 ms), which sits under every dispatch gate those backends define (≥1024 tokens / ≥32768 M·N·K / tile M covering only prefill), and the codec half of the pipeline is conv, col2im, gather and elementwise ops — not MUL_MAT — so it would fall back to CPU op-by-op. The NPU case for this host is therefore not "run Qwen3-TTS on the NPU"; it is "run a vendor-compiled AR TTS graph on the NPU", which is exactly the NeuTTS-Air spike.

ANSWER C — would qwentts.cpp build for the gfx1151 iGPU? Yes, via Vulkan, not HIP

• Upstream ships the build scripts and claims the backends: buildvulkan.sh ("AMD/Intel GPU (Vulkan)"), buildcuda.sh, buildsycl.sh, buildcpu.sh, buildall.sh (all backends, runtime dynamic loading), and README line 6 "runs on CPU, CUDA, Metal, Vulkan". CMake knobs GGML_VULKAN / GGML_HIP / GGML_SYCL are the same ones the fleet's Containerfile turns OFF, so the engine rebuild is a flag change: -DGGML_VULKAN=ON plus --device Vulkan0 on the same command line the unit runs today.
• The codec's exotic ops are covered by the Vulkan backend, which is the thing that decides whether a GPU build is coherent: vulkan-shaders/col2im_1d.comp, snake.comp, swiglu*.comp, cumsum*.comp exist and supports_op handles GGML_OP_COL2IM_1D / CONV_TRANSPOSE_1D / CUMSUM. By contrast COL2IM_1D appears in no HIP, OpenCL, WebGPU, Hexagon or CANN source — a HIP (ROCm) build would shard the ConvNeXt/DAC transposed convs back to the CPU per op. On an AMD iGPU the portable path is RADV/Vulkan.
• --device also accepts a multi-device split (-dev-style naming is ggml's), and the runtime's documented FP16 guard (--clamp-fp16) exists for sub-Ampere CUDA accumulation; nothing in the tree is NVIDIA-specific.
• Unverified by me: any vendor statement pairing this fork with gfx1151. Supporting evidence that the hardware/driver combination is viable: this very host already runs a Vulkan endpoint on the gfx1151 iGPU (fleet ground truth, 2026-10-19), and Mesa's Vulkan implementation is officially supported on Radeon Software for Linux (AMD release notes RN-AMDGPU-UNIFIED-LINUX-25.20.3). A gfx1151 Vulkan build of this fork is a build-and-measure spike, not an unsupported port.
• The real constraint is the one already in the fleet docs: the iGPU belongs to another endpoint, and GPU+NPU concurrency has hung Strix Halo machines. An iGPU TTS build competes with the text endpoints for the same silicon; the NPU route's only genuine advantage is that it does not (the XDNA backend authors make the same argument: "Use NPU when the iGPU is busy … no GPU contention").

---

4. What any replacement must reproduce, and what the voices cost

Endpoint contract in production today (from the live unit/env, not the doc): tts-server --model qwen-talker-1.7b-base-Q4_K_M.gguf --codec qwen-tokenizer-12hz-Q8_0.gguf --alias frank-tts-engine --host 0.0.0.0 --port 8880 --lang auto --max-batch 1, no published port (the façade on :7772 dials the private bridge name), GET /health, GET /v1/models, GET|POST /v1/audio/voices, DELETE /v1/audio/voices/{name}, POST /v1/audio/speech with response_format wav|pcm (different audio at equal seed) and optional seed, max_new_tokens, temperature, top_k, top_p, repetition_penalty, subtalker_*. Voice registry is RAM-only, so boot warming is mandatory, and --alias keeps the quantization choice out of every consumer's config.

Live voice census on Dankclaw (2026-10-19, /home/dank/voices/): three packs (narrator1..3), each carrying reference.wav (24 kHz mono crop), reference16k.wav, manifest.json (keys: name, kind, generated_utc, pipeline, source, forbidden_region_guard, silence_scan, crop, loudnorm, audio, acceptance, acceptance_passed) and ffmpeg analysis artifacts. No .spk and no .rvq exist on the live host — the repo's build-voice-pack.sh treats them as required-but-they-are-not-there, so today the engine re-extracts the x-vector (and the ICL codes when a transcript is supplied) from the WAV at registration time.

That is the portable part, and it is the same shape AMD's NeuTTS flow wants (encode_reference(reference.wav) + a reference transcript): the fleet's voice identity is the WAV plus its transcript, processed at request/registration time by the model's own speaker encoder + codec — never a trained per-voice artifact. Qwen3-TTS cloning is inference-time conditioning (generate_voice_clone(ref_audio=…, ref_text=…, x_vector_only_mode=…); create_voice_clone_prompt exists purely to avoid recomputing prompt features). So any replacement that consumes a WAV (+transcript) keeps all three narrators alive with no re-derivation, and if it wants precomputed latents instead, the LiteRT precedent shows what that looks like: a 4 KB x-vector .npy per voice, extracted offline (extract_speaker_embedding.py) — for the 1.7 B base checkpoint that vector is 2048-dim (qwen3-tts.spk_enc.embedding_length = 2048 in the shipped GGUF; the LiteRT 0.6 B bundle ships a 1024-dim one).

One trap for a mixed-runtime future: .spk/.rvq files, if they are ever written, are engine-specific encodings (ECAPA x-vector layout; 11-bit-packed RVQ stream) and are not interchangeable between runtimes. Keep reference.wav as the pack's source of truth and regenerate anything derived per engine — which is what the NeuTTS re-clone plan does anyway.

5. Sources

Runtime/payload (all read locally at pinned versions, 2026-10-19):

• github.com/ServeurpersoCom/qwentts.cpp @ 5916bf7c79dbb4e66fc059cdc507a79a23a28bb4 (commit date 2026-10-16; also upstream HEAD today): README.md, docs/ARCHITECTURE.md (860 lines), src/*, tools/*, docs/DOCKER.md, buildvulkan.sh/buildcpu.sh/buildall.sh.
• submodule ServeurpersoCom/ggml @ cc5df49a6b878db2b68b9a7b45976e279594627d (2026-10-16); delta vs ggml-org/ggml master a05c992 (2026-10-15) = ahead 2 / behind 0, per https://github.com/ServeurpersoCom/ggml/compare/ggml-org:master...HEAD ; include/ggml.h identical to upstream https://raw.githubusercontent.com/ggml-org/ggml/master/include/ggml.h .
• Live payload bytes: /home/dank/models/Qwen3-TTS-GGUF/qwen-talker-1.7b-base-Q4_K_M.gguf (1,219,245,248 B) and qwen-tokenizer-12hz-Q8_0.gguf (291,150,624 B), headers parsed from the host; sizes/sha256 match scripts/download-frank-tts-engine-qwentts.sh.
• Live units/env/packs: dankclaw/voiceservice/{Containerfile.engine,tts-engine-start.sh,tts-engine-warm.sh,quadlet/*}, scripts/frank-tts-engine.env, scripts/frank-tts.env, /home/dank/voices/*.

Model provenance:

• https://huggingface.co/api/models/Qwen/Qwen3-TTS-12Hz-1.7B-Base (+ CustomVoice, VoiceDesign, 0.6B variants, Qwen3-TTS-Tokenizer-12Hz), apache-2.0, last-modified 2026-01-23 / 2026-01-29; safetensors param counts 1,928,677,440 (BF16, incl. embedded speech_tokenizer/) and 170,557,441 (F32 codec).
• https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base/raw/main/README.md (clone API: generate_voice_clone(ref_audio, ref_text, x_vector_only_mode), create_voice_clone_prompt; reference stack = PyTorch + device_map="cuda:0" + bf16 + flash-attn-2, Python 3.12 recommended).
• https://github.com/QwenLM/Qwen3-TTS.

Portability precedents:

• https://huggingface.co/xkos/Qwen3-TTS-12Hz-1.7B-ONNX (2026-03-03) · https://github.com/zhangluoyang/qwen3-tts-onnxruntime · https://huggingface.co/Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO (2026-03-22) · https://huggingface.co/aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8 (2026-03-18) · https://huggingface.co/litert-community/Qwen3-TTS-12Hz-0.6B-Base (2026-10-17) incl. legacy/README.md (graph splits, MTP fold, codec partA/partB, RTF table).

AMD NPU / ggml backend status:

• https://api.github.com/repos/ggml-org/llama.cpp/git/trees/master?recursive=1 (no XDNA/AIE/Vitis backend, 2026-10-19) · https://github.com/ggml-org/llama.cpp/issues/21725 (open since 2026-04-10) · https://github.com/ggml-org/llama.cpp/issues/1499 (closed stale 2025-04-07).
• https://github.com/Cyronius/ggml-xdna (prefill-only, gates, Linux requirements, results measured 2026-10-05/06 on HX 370) · https://github.com/BrandedTamarasu-glitch/OllamaAMDNPU (XDNA2 ggml backend on Strix Halo, phases 1–12, "use Vulkan exclusively", measured on Ryzen AI MAX 385).
• https://github.com/amd/RyzenAI-SW (main, 669 paths, 2026-08-18): audio content is ASR only (Demos/ASR/Whisper, Demos/ASR/Parakeet-TDT, Transformer-examples/ASR/Whisper-AI); no TTS example anywhere in the official Ryzen AI Software tree.
• https://ryzenai.docs.amd.com/en/latest/relnotes.html (Ryzen AI SW 1.8.0: ONNX-format models, ONNX Runtime as the loader, power-of-two QDQ/QOP quantization).
• Mesa/RADV on AMD iGPUs: https://www.amd.com/en/resources/support-articles/release-notes/RN-AMDGPU-UNIFIED-LINUX-25-20.3.html (Mesa Vulkan driver officially supported).

Status: COMPLETE for the assigned slice (payload architecture, runtime architecture, portability). [INFERENCE] marks: none above except the Mesa-version caveat in ANSWER C, which is flagged there.