Merge origin/main into docs/radiance-prefill-decode-census

Two conflicts, both from the 2026-10-19 vault compaction on main:

* frontmatter description — main shortened it to satisfy OKF DESC
  (max 120 chars); this branch's longer wording would have failed the
  seam. Kept main's shape and folded the new §9 in by naming
  step-shape attribution instead of profiler attribution.
* the Related bullet for [[vLLM Porting and Compile Triage]] — that
  page was destroyed on main, so the link is dropped rather than
  resurrected; this branch's [[frank-archon]] row is kept.

Index rows regenerated with scripts/gen-vault-index.py.

report/npu-tts-runtime-landscape-2026-10-19.md

@@ -0,0 +1,319 @@
+# XDNA2 NPU inference runtimes on Linux (Strix Halo) — can any of them serve TTS?
+
+Research slice, 2026-10-19. Worker: `NpuRuntimeLandscape`. Research only, no code.
+Question: every way to execute model inference on an AMD XDNA2 NPU on Linux as of
+2026-10, and whether any of them can serve a speech-synthesis model.
+
+> **Status, same night — the spike that followed this slice settled it.** NO-GO on the
+> NPU (the loader declines the graph, not the silicon); the actionable door is the gfx1151
+> iGPU behind `-DGGML_VULKAN=ON`. Measured contract, exoneration chain and host gates:
+> `tts-npu-spike-verdict-2026-10-19.md`. That page supersedes §8–§9 below.
+
+Convention: vendor claims carry URL + the version/date they hold for.
+**[host]** = Dankclaw state verified on the live machine 2026-10-19 (Main), not a
+vendor claim. `[INFERENCE]` = my deduction from the cited artifacts.
+
+## 0. Answers to the three questions Main ranked
+
+**(1) A runtime that already runs TTS on XDNA2 today: it exists at AMD, not on our
+host.** Ryzen AI Software **1.8.0** (docs `latest` == 1.8.0; released 2026-07-23)
+lists `neutts-air` and `neutts-nano` as model type **TTS**, variant
+*full-optimized* (NPU-only) — <https://ryzenai.docs.amd.com/en/latest/llm_list.html>;
+release note "New TTS model support: neutts-air, neutts-nano" —
+<https://ryzenai.docs.amd.com/en/latest/relnotes.html>. Pre-compiled artifacts are
+published: `amd/neutts-air_rai_1.8.0_npu_4K` (arch `qwen2`, lastModified
+2026-07-23, Apache-2.0, "AWQ / Group 128 / Asymmetric / BFP16 activations / UINT4
+Weights") and `amd/neutts-nano_rai_1.8.0_npu_4K` (arch `llama`).
+**Neither artifact is a TTS engine.** Both file lists are LM-only (`model.onnx`,
+`genai_config.json`, BPE tokenizer, AIE-ML compiler metastate: `dd_metastate_*`
+`.ctrlpkt/.super/.state`, `cache/txn_bins.zip`) with **no codec, no vocoder, no
+reference-audio encoder**. [INFERENCE from HF file lists]
+
+**(2) FastFlowLM: no TTS, and we cannot add one** — see §2. Halogen: no TTS entry
+**[host]**. So "something installed already serves TTS on the NPU" is FALSE.
+
+**(3) Our own ONNX graph compiled for the NPU on Linux:** possible for the plain
+ONNX flows (they compile on Linux), **impossible for a new LLM/OGA model** — model
+generation is Windows-only in 1.8.0 — and gated by an operator table where BF16
+lacks Softmax/LayerNorm/ReLU/Gelu (see §4).
+
+## 1. Landscape: every way to reach the XDNA2 array on Linux
+
+| # | Path | License | Strix Halo + Linux, 2026-10 | Speech synthesis |
+|---|------|---------|------------------------------|------------------|
+| 1 | Ryzen AI SW → ONNX Runtime + Vitis AI / Ryzen AI EP; compile INT8 (`X2`) or BF16 (`VAIML`) ONNX at session creation | AMD proprietary (tarball) | supported: our device ID/rev is in AMD's STX table; compile+run works on Linux (§3, §4) | user-authored ONNX only; no TTS recipe/demo |
+| 2 | Same EP driven by **OGA** (`onnxruntime-genai`), "full fusion" NPU-only LLM flow; runner = AMD `model_benchmark` / OGA C++ API | AMD + MS OGA | Linux: **vendor-precompiled models only**, no hybrid flow | **neutts-air / neutts-nano = TTS** (LM half) |
+| 3 | **FastFlowLM** `flm` | MIT + free proprietary kernels | Strix Halo "Supported"; Ubuntu 24.04/25.10/**26.04**, Arch | none (ASR input only) |
+| 4 | **IRON / mlir-aie** + Peano → hand-written AIE-ML design → XRT/`amdxdna` | Apache-2.0 + LLVM exception | explicitly documented "AMD Ryzen™ AI on Linux"; same driver/XRT stack | anything, hand-written; nothing ships TTS |
+| 5 | Vendor-curated AIE-ML graph sets (halogen `HALOGEN_NPU_MODELS`, FLM packs) | vendor | present **[host]** | none **[host]** |
+| 6 | Windows-only: DirectML, Ryzen AI Windows EP (`ryzenai-llm`), Studio Effects, `whispercpp` NPU | — | not usable | (per Lemonade matrix: Windows only) |
+| 7 | ggml/llama.cpp/**qwentts.cpp**, whisper.cpp, sherpa-onnx, Piper | — | no XDNA backend in ggml/ORT-mainline | CPU / iGPU only |
+| 8 | IREE, TVM, ONNX Runtime mainline EPs (OpenVINO = Intel NPU, MIGraphX/ROCm = AMD GPU) | — | no XDNA2 target; AMD's NPU EP is only "Xilinx Vitis-AI (preview)" in the ORT EP list and ships inside AMD's own build | n/a |
+
+Sources: §2–§6 below.
+
+## 2. FastFlowLM (`/usr/bin/flm` 1.0.1 **[host]**) — catalog + HTTP surface
+
+* Repo moved into AMD's ROCm org **2026-08-11** (`github.com/ROCm/FastFlowLM`,
+  v1.0.0). "Supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo,
+  Kraken, and Gorgon Point)."
+  <https://github.com/ROCm/FastFlowLM> (README fetched 2026-10-19).
+* Linux support landed 2026-03; distros: Ubuntu 24.04, 25.10, **26.04**, Arch,
+  generic. Needs NPU firmware ≥ 1.1.0.0, kernel **7.0+** with `amdxdna` (or
+  `amdxdna-dkms`), XRT stack (`ppa:lemonade-team/stable` → `libxrt-npu2`,
+  `amdxdna-dkms`); `flm validate` reads the DRM path (`/dev/accel/accel0`), `flm
+  run` goes through **XRT** and fails with `No such device with index '0'` unless
+  `xrt-smi examine` lists the NPU; `memlock` must be raised.
+  <https://fastflowlm.com/docs/install_lin/> (page dated 2026-03-05, fetched
+  2026-10-19). Static build bundles **XRT v2.21.75** — the same XRT series as the
+  distro package on the host **[host]**.
+* HTTP surface: `flm serve <model>` → OpenAI-compatible REST, **port 52625**, no
+  auth. Maintained endpoints: `v1/models`, `v1/chat/completions`,
+  `v1/audio/transcriptions`, `v1/embeddings`. **There is no
+  `/v1/audio/speech`.**
+  <https://fastflowlm.com/docs/instructions/server/openapi/>
+* Catalog 2026-10-19 (<https://fastflowlm.com/docs/models/>): LLaMA, DeepSeek,
+  Qwen, Gemma, MedGemma, TranslateGemma, gpt-oss, LiquidAI/LFM, Phi, Nanbeige,
+  **Whisper — card states "Type: Speech-to-Text (ASR)"**
+  (<https://fastflowlm.com/docs/models/whisper/>), EmbeddingGemma, SmolVLA,
+  Hy-MT2, MiniCPM. No speech-synthesis entry.
+* Engines are **prebuilt, closed, downloaded from Hugging Face per architecture**
+  ("Internet access to HuggingFace is required to download the optimized model
+  kernels"). The only in-tree engine build is Gemma-4E
+  (`cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON`).
+  → **A new TTS architecture cannot be added to `flm` by us.**
+
+## 3. AMD Ryzen AI Software 1.8.0 on Linux — install source, device support
+
+* **Install channel (important correction to the pypi.amd.com evidence).** The pip
+  index is Windows-only for the EP runtime (`ryzenai_dynamic_dispatch-1.7.0-…
+  win_amd64.whl`, `onnxruntime_vitisai-1.23.2-…win_amd64.whl`; `flexmlrt` has a
+  single **linux_aarch64** wheel) — but AMD's Linux stack is not on pip. It ships
+  as a tarball that is, as of 2026-10-19, **publicly reachable without an AMD
+  account**: `https://download.amd.com/opendownload/RyzenAI/1.8.0/ryzen_ai-1.8.0.tgz`
+  → HTTP 206 on a range probe, `Content-Range: bytes 0-8/**7819506855**`
+  (**≈7.8 GiB**, magic `1f 8b 08` = gzip). (The docs link it behind the
+  EULA form `account.amd.com/...ryzenai-eula-public-xef.html`; the CDN path
+  serves it directly. Verified by me with curl 2026-10-19. I did not finish
+  downloading 7.8 GiB, so the tarball's *contents* list is unverified —
+  `[INFERENCE]` only where noted.)
+* Driver/XRT package (public, 90,016,137 bytes, `application/zip`):
+  `https://download.amd.com/opendownload/RyzenAI/Driver/RAI_1.8_Linux_NPU_XRT.zip`
+  contains `xrt_202620.2.25.37_24.04-amd64-{base,base-dev,npu}.deb` +
+  `xrt_plugin.2.25.260102.56.release_24.04-amd64-amdxdna.deb` (verified by
+  download + unpack). **The plugin deb ships firmware for
+  `/usr/lib/firmware/amdnpu/17f0_11/npu.dev.sbin`** — exactly the directory our
+  NPU loads **[host]** — plus `1502_00`, `17f0_10`, `17f1_10`, `17f1_13`,
+  `libvxdna.so`, `libxrt_driver_xdna.so.2.25.260102.56`, and the DKMS driver
+  source tarball. **No xclbin/PDI and no compiler binary in this package**: it is
+  driver + XRT shim + firmware only.
+* **Our device is officially STX.** AMD's APU table
+  (<https://ryzenai.docs.amd.com/en/latest/app_development.md#apu-types>):
+  `0x1022:0x17F0 rev 0x11 → STX` (0x00/0x10 also STX, 0x20 = KRK, `0x1502` rev 0
+  = PHX/HPT). Release notes: "Strix and **Strix Halo** as STX", and the STX/KRK
+  column enables CNN INT8, CNN BF16, NLP BF16, LLM (OGA).
+* Vendor driver source (same deb, `share/amdxdna/amdxdna.tar.gz`, source dated
+  2026-06-29) maps `0x17f0 rev 0x11 → dev_npu5_info`, `.fw_path =
+  "amdnpu/17f0_11/"`, `default_vbnv = "RyzenAI-npu5"`, and the revision→name table
+  has `{ AIE2_DEV_REVISION_HALO, "NPU Strix Halo" }`; `xrt-smi examine` therefore
+  reports the device as **NPU Strix Halo**.
+* Linux install recipe (1.8.0 docs
+  <https://ryzenai.docs.amd.com/en/latest/linux.html>): Ubuntu 24.04 packages,
+  **Python 3.12** (`apt install python3.12 python3.12-venv`),
+  `libboost-filesystem1.74.0`, dkms; then `install_ryzen_ai.sh -a yes -p <venv>`;
+  `LD_LIBRARY_PATH=…:${RYZEN_AI_INSTALLATION_PATH}/onnxruntime/lib/`;
+  `quicktest.py` "picks up a simple CNN model, **compiles it** and runs on AMD's
+  Neural Processing Unit" — i.e. **compilation runs on Linux** for the non-LLM
+  flows. "Linux currently supports NPU only flow" (no iGPU EP).
+  Host friction: our box is Ubuntu 26.04 + **Python 3.14** **[host]**, so the venv
+  must be built with a 3.12 interpreter and the packages are 24.04-built
+  `[INFERENCE]`.
+* LLM flow on Linux
+  (<https://ryzenai.docs.amd.com/en/latest/llm_linux.html>): prequantized
+  ready-to-run models from HF, `deployment/lib`,
+  `RYZENAI_EP_PATH=$PWD/deployment/lib/libonnxruntime_providers_ryzenai.so`,
+  run via the `model_benchmark` binary; **"Linux does not support Hybrid flow"**;
+  **"Model generation is not supported in current release"** (repeated in
+  relnotes 1.8 Known Issues: "Models generated on Windows are compatible with
+  Linux"); needs `memlock unlimited`.
+  Sample throughput in that doc: **49.1 tokens/s decode**, 753 tokens/s prefill
+  (Phi-3.5-mini, 4K model).
+* Third-party confirmation of the OS split: Lemonade's backend matrix
+  (<https://raw.githubusercontent.com/lemonade-sdk/lemonade/main/README.md>,
+  fetched 2026-10-19) — `flm`/npu/XDNA2 = **Windows, Linux**;
+  `ryzenai-llm`/npu = **Windows only**; `whispercpp`/npu (STT) = **Windows only**;
+  text-to-speech engines `kokoro` and `acestep` = metal / cuda / vulkan / rocm,
+  **no npu device at all**. Lemonade is the "Server Interface" layer of AMD's own
+  Ryzen AI stack (docs index), so this matrix is the vendor-sanctioned view.
+
+## 4. Can we compile our own graph on Linux? (the no-recipe path)
+
+Mechanics from <https://ryzenai.docs.amd.com/en/latest/modelrun.md> (1.8.0):
+ONNX only, opset 17 recommended; the EP compiles on first `InferenceSession` and
+caches (VitisAI EP cache, or ORT **EP context cache** — `ep.context_enable`, and
+"precompile the EP context model using Python and deploy it using a C++ program"
+is explicitly supported); EP auto-partitions unsupported nodes to CPU;
+BF16 = `target VAIML` (`preferred_data_storage: unvectorized` for transformers),
+BF16 **STX/KRK-or-newer only**; INT8 = `target=X2` with XINT8/A8W8/A16W8 and
+"general asymmetric quantization … to enable third-party quantized models";
+**deployment EP cannot compile BF16 on the fly** — ship a pre-compiled model
+(<https://ryzenai.docs.amd.com/en/latest/app_development.md>);
+cache dirs are not portable across EP or NPU-driver versions.
+
+Operator coverage is the hard gate
+(<https://ryzenai.docs.amd.com/en/latest/ops_support.md>, "operators supported in
+Ryzen AI 1.8.0"; "Y" = broad coverage, some configurations may still fail):
+
+* Good for TTS front-ends/vocoders: `ConvTranspose` **Y in BF16/A16W8/A8W8**
+  (absent only from XINT8), `Conv`, `Resize`, `Upsample`, `Pad`, `LSTM` (BF16),
+  `CumSum` (BF16/XINT8), `STFT` (XINT8 only).
+* **Blockers for a decoder-style transformer in the plain flows:** in **BF16** the
+  table has **no Softmax, no LayerNormalization, no Relu/Gelu**; LayerNorm is
+  XINT8-only and Softmax is A16W8/A8W8/XINT8-only. So attention either lives in
+  the **LLM/OGA flow** (whose op set is exactly
+  `SimplifiedLayerNormalization`, `SkipSimplifiedLayerNormalization`, `MatMulNBits`
+  (W4ABF16/W4ABFP16), `RotaryEmbedding`, **`GroupQueryAttention`**, `Sigmoid`,
+  `Mul`, `Add`) or must be decomposed into a quantized INT8 flow.
+* Missing entirely: `GRU`, `Trilu`, `OneHot`, `EyeLike`, `Multinomial`,
+  `NonMaxSuppression`, `CastLike`, `MelWeightMatrix`/`MelFilterGroup`,
+  `RMSNormalization` → graphs that emit them must be rewritten before they
+  partition onto the NPU. [INFERENCE from absence in the table.]
+* Quantization is a separate tool, not in the driver package: **AMD Quark**
+  (`quark.torch` / `quarkonnx`), which documents `ConvTranspose2d`, `conv1d`,
+  softmax/LayerNorm/BFP quantization (relnotes 1.3/1.2/1.1/0.9) — Quark is *not*
+  on pypi.org under the obvious names **[host probe]** and ships from AMD's
+  gated/pip-private channel.
+* **New LLM-shaped model (our Qwen-Talker analogue): Windows-only today.**
+  OGA model generation is not available on Linux in 1.8.0 (§3). So a *new*
+  speech-LM cannot be self-served onto the NPU on Linux without AMD (or a
+  Windows box) in the loop.
+* Practical shape cost: the neutts artifacts carry **one compiled variant per
+  padded sequence length** (128/256/512/768/1024/1536/2048/2304/3072/4096 for
+  prefill; 256…4096 for token-phase attention mask) — i.e. per-shape compile
+  artifacts are the norm for the LLM flow. [INFERENCE from the HF file list.]
+
+## 5. Does the AMD TTS artifact actually produce speech? (cloned-voice survival)
+
+NeuTTS is **LM + separate neural codec** ("Simple LM + codec architecture"),
+clone from ~3 s of reference audio, and the codec is **NeuCodec (50 Hz, single
+codebook)**: `neuphonic/neucodec` (PyTorch, encode+decode),
+`neucodec-onnx-decoder{,-int8}` (**decode only**), "Decoder-only (ONNX) codecs
+require **pre-encoded references** — these ship with the repo for all bundled
+voices." Phoneme input; 2048-token context ≈ 30 s of audio; NeuTTS-Air ~360 M
+active / 552 M with embeddings; upstream benchmarks are **CPU-only llama.cpp**
+(119 tok/s prefill/decode mix on a Ryzen 9 HX 370) and explicitly exclude the
+codec. <https://raw.githubusercontent.com/neuphonic/neutts/main/README.md>
+(fetched 2026-10-19); `neuphonic/neutts-air` HF metadata (lastModified
+2026-08-26, `qwen2`, GGUF + safetensors).
+
+Consequences for the fleet spike:
+
+1. AMD's NPU artifact = LM only; **waveform decode and reference encoding are CPU
+   work either way**. Our `reference.wav` voices survive only if we run the
+   NeuCodec **encoder** (PyTorch, per the codec table there is no ONNX encoder)
+   once per voice and then feed pre-encoded codes. That is a one-time offline
+   step per voice; after that the compiled graph takes codes/prompt prefix as
+   ordinary inputs, so nothing about the compiled graph forbids a caller-supplied
+   speaker prefix. `[INFERENCE]`
+2. Real-time math: 50 Hz codec means **≥50 tokens/s decode** for 1× RT. AMD's own
+   Linux sample output for a 4K Phi-3.5-mini is **49.1 tok/s** — so budget for
+   roughly RT-1× on one 4K "full fusion" graph, before the codec's CPU cost.
+   `[INFERENCE — different model; must be measured.]`
+3. Host RAM: the same doc reports **peak working set 3.54 GB** for that 4K model.
+   NeuTTS-Air is ~552 M params at W4A-BFP16, so expect a smaller footprint than
+   3.5 GB `[INFERENCE]`; still non-trivial against 14 GiB available **[host]**.
+4. AMD ships **no TTS demo, runner, or server** on any OS: the examples page has
+   only CNN/BF16/DistilBERT/super-resolution/OGA-LLM/VLM/RAG
+   (<https://ryzenai.docs.amd.com/en/latest/examples.md>); the RyzenAI-SW repo
+   tree (669 paths, `main`, fetched 2026-10-19) contains speech demos **only under
+   `Demos/ASR/{Whisper,Parakeet-TDT}` and `Transformer-examples/ASR/Whisper-AI`**
+   — no TTS directory. Serving TTS therefore means writing our own OGA host
+   (or driving `model_benchmark`-style C++ code) plus phonemizer + NeuCodec glue.
+
+## 6. Two graphs on one NPU, and the iGPU hazard
+
+* **Concurrency is a first-class driver feature.** The vendor `amdxdna` source
+  (in `xrt_plugin.2.25.260102.56`, 2026-06-29) sets `.hwctx_limit = 16` for
+  npu4/npu5 — i.e. **16 hardware contexts** on our device — and allocates each
+  context a set of AIE columns with QoS `opc = col_opc (4096) × num_col`
+  (`aie2_ctx.c`, `aie2_solver.h`), with `frame_boundary_preempt` /
+  `fill_preempt_slot` in `aie2_pci.h` (firmware preemption at frame boundaries).
+  So halogen's armed models and a TTS graph can hold separate contexts
+  simultaneously; they contend for columns and OPC share, they are not mutually
+  exclusive.
+* AMD's user-visible history agrees: 1.4 "Support for up to **16 hardware
+  contexts** on Ryzen AI 300 series"; 1.5 "Application concurrency: improves the
+  resource distribution across applications"; 1.3 "Idle detection and suspension
+  of contexts", "Rebalance feature for AIE hardware resource optimization";
+  1.0 known issue "Support for multiple applications is limited to up to eight"
+  (all in <https://ryzenai.docs.amd.com/en/latest/relnotes.html>).
+* **Package conflict, the real risk** [INFERENCE]: the vendor stack wants AMD's
+  XRT 2.25.37 + `xrt_plugin…amdxdna` deb **[host: distro `libxrt-npu2
+  1:2.21.75+dfsg-4` installed]**, and firmware must match the loaded `amdxdna.ko`
+  — the vendor README warns that a stale/mismatched `npu.sbin` causes
+  `ERT_CMD_ABORT`/mailbox timeouts, and that a distro **in-tree** `amdxdna` older
+  than the XRT SHIM makes `GET_ARRAY`/`QUERY_TELEMETRY` return
+  `EOPNOTSUPP`/`EINVAL` (<https://github.com/amd/xdna-driver> README,
+  fetched 2026-10-19). Installing AMD's debs over the distro XRT changes the ABI
+  under FastFlowLM/halogen, so the spike must decide the XRT version **once**, for
+  the whole host.
+* **iGPU / fabric clock:** no vendor document supports a fabric-clock pin, and no
+  vendor document addresses NPU+iGPU concurrency hangs. AMD's Strix Halo
+  optimization doc (ROCm docs, dated 2026-02-20, and its successor "AMD RDNA3.5
+  system optimization", ROCm 10.1.0) covers GPUVM/GART/GTT, the TTM `pages_limit`
+  knob, `amd-ttm`, BIOS VRAM sizing, and minimum kernel versions
+  (<https://rocm.docs.amd.com/en/docs-7.2.0/how-to/system-optimization/strixhalo.html>,
+  <https://rocm.docs.amd.com/en/latest/reference/system-optimization/rdna3-5.html>).
+  The `pp_dpm_fclk` level-7 pin and the GPU+NPU hang observation stay
+  fleet-observed **[host]**, i.e. `[INFERENCE]` territory — no citation exists.
+
+## 7. Paths I checked and ruled out (so nobody re-walks them)
+
+* **Windows-only doors**: DirectML; Ryzen AI Windows EP; Lemonade's
+  `ryzenai-llm` and `whispercpp`-NPU engines (matrix above); AMD's OGA model
+  generation. Not usable on Dankclaw.
+* **Mainline ONNX Runtime** has no AMD-NPU EP: its EP summary lists AMD ROCm and
+  AMD MIGraphX (GPU) and "Xilinx Vitis-AI (*preview*)" only
+  (<https://onnxruntime.ai/docs/execution-providers/>, 2026-10-19); the working
+  EP is AMD's own build (`libonnxruntime_providers_ryzenai.so` in the tarball's
+  deployment dir, per llm_linux).
+* **ggml / llama.cpp / qwentts.cpp**: no XDNA backend → the current engine
+  (`tts-server` on GGUF, CPU-only build **[host]**) cannot be "turned on" for the
+  NPU; migration means a different runtime, not a flag.
+* **`flm`**: catalog/engines are closed per-architecture (§2) — the cheapest door
+  by install cost, and it is not a door for TTS.
+* **IRON / mlir-aie** is the only open toolchain that documents running on the
+  Ryzen AI NPU from Linux ("A close-to-metal Python API for programming AMD Ryzen™
+  AI NPUs, built on an open-source MLIR-based compiler toolchain"; "Getting
+  Started for AMD Ryzen™ AI on Linux", Ubuntu 24.04/24.10, XRT from
+  `ppa:amd-team/xrt`: `libxrt2 libxrt-npu2 … amdxdna-dkms`, `render` group
+  required) — <https://raw.githubusercontent.com/Xilinx/mlir-aie/main/README.md>,
+  fetched 2026-10-19. This is how FLM's kernels were made ("Low-level kernels
+  optimized using IRON + AIE-MLIR", FLM README) — i.e. real, but it is
+  hand-written per-op tile programming, not a model compiler: writing a TTS model
+  here is a multi-month engineering project, not a migration.
+* **Embedded** Ryzen AI P100 support in the same Linux doc is irrelevant to STX.
+
+## 8. Recommendation implied by the above (for Main's spike)
+
+Spike the AMD path as decided, with these constraints planned in: Python 3.12 venv
++ 24.04-built wheels on Ubuntu 26.04; one host-wide XRT decision (AMD 2.25.37 vs
+distro 2.21.75) with firmware/driver pairing checked via `xrt-smi examine` →
+"NPU Strix Halo"; 7.8 GiB tarball staged off the public CDN URL (§3); pre-encode
+each `reference.wav` with the **PyTorch** NeuCodec encoder because the ONNX codec
+is decoder-only; keep phonemizer + codec on CPU; measure decode tokens/s against
+the 50 Hz budget and peak RSS against the ~14 GiB headroom; expect to write the
+serving shell yourself (AMD ships `model_benchmark`, no TTS demo).
+
+## 9. Loose ends (not verified; I stopped on Main's stand-down)
+
+* Contents of `ryzen_ai-1.8.0.tgz` (7.8 GiB) — whether the Linux tarball contains
+  the AIE-ML compiler (`vaiml`, `aiecompiler_client`) for the *BF16/INT8* flows
+  and which Python wheels it carries. My statement that non-LLM compilation works
+  on Linux rests on AMD's own Linux doc ("quicktest.py … compiles it and runs on
+  … NPU"), not on the package contents. Download was killed mid-flight.
+* Whether AMD's `neutts` artifacts run correctly under the Linux `model_benchmark`
+  runner (the TTS rows are listed without OS qualifiers; no Linux TTS run is
+  documented anywhere in 1.8.0 docs).
+* Exact AIE column count/total `ncols` exposed for Strix Halo (driver reads it
+  from AIE metadata: `ndev->total_col = min(XRS_MAX_COL, ndev->aie.metadata.cols)`).

report/qwen3-tts-payload-portability-2026-10-19.md

@@ -0,0 +1,438 @@
+# Qwen3-TTS payload + qwentts.cpp runtime: architecture and portability
+
+Research deliverable (no code). Slice: *what the fleet's TTS payload actually is, and how
+portable it is off the CPU ggml path.* Companion slice: `NpuRuntimeLandscape` (what NPU
+runtimes exist on Dankclaw).
+
+Evidence date: 2026-10-19. Everything below is grounded in:
+
+- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
+  (2026-10-16, upstream HEAD on 2026-10-19) and its ggml submodule
+  `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16).
+- The two GGUF files as they sit on Dankclaw (`/home/dank/models/Qwen3-TTS-GGUF/`), read
+  byte-for-byte from the live host.
+- Upstream Qwen3-TTS model cards / release material.
+
+Status: COMPLETE for the assigned slice. ANSWER C was confirmed on the live host the same
+night — the running `frank-tts-engine` container carries `libggml-base.so.0.27.0` +
+`libggml-cpu.so.0.27.0` and nothing else, and our own
+`dankclaw/voiceservice/Containerfile.engine` writes `-DGGML_VULKAN=OFF` out explicitly, so
+the iGPU change is a flag in a file the fleet owns. Verdict, measured contract and the NPU
+exoneration chain: `tts-npu-spike-verdict-2026-10-19.md`.
+
+---
+
+## 1. What the payload is
+
+Two files, both read byte-for-byte off the live host (`/home/dank/models/Qwen3-TTS-GGUF/`,
+owned `dank:dank`, mtimes 2026-10-18 18:31), both from
+`https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF` (community converter, not Qwen):
+
+| file | bytes | GGUF | tensors | arch string | file_type |
+|---|---|---|---|---|---|
+| `qwen-talker-1.7b-base-Q4_K_M.gguf` | 1,219,245,248 | v3, kv=64 | 480 | `qwen3-tts` | `Q4_K_M` |
+| `qwen-tokenizer-12hz-Q8_0.gguf` | 291,150,624 | v3, kv=51 | 398 | `qwen3-tts-tokenizer` | `Q8_0` |
+
+`general.name` of the talker file is `Qwen3-TTS-12Hz-1.7B-base`; the header sizes add up
+to the file sizes (sum of tensor bytes + header + alignment padding), so nothing in these
+files is unaccounted for.
+
+### 1.1 Talker file (480 tensors)
+
+Three models in one container, per `docs/ARCHITECTURE.md` §GGUF layout
+(qwentts.cpp @ pinned commit) and confirmed by the tensor census:
+
+| prefix | tensors | bytes | what it is |
+|---|---|---|---|
+| `talker.*` | 316 | 1,061.9 MB | 28-layer Qwen3 decoder (hidden 2048, FFN 6144, 16 Q heads / 8 KV heads, head_dim 128, RoPE θ=1e6, RMS ε=1e-6, ctx 32768), text stream + codec stream summed, `codec_head` → 3072 codebook-0 logits |
+| `code_pred.*` | 88 | 114.8 MB | 5-layer Qwen3 MTP head (hidden 1024, FFN 3072, 15 acoustic codebooks × 2048), own KV cache |
+| `spk_enc.*` | 76 | 36.6 MB | ECAPA-TDNN speaker encoder (x-vector, base checkpoints only; mel → conv0 → 3 SE-Res2Net → MFA → ASP → 2048-d vector) |
+
+Dtype split (exact tensor byte sums): `q4_K` 250 tensors / 864.1 MB, `q6_K` 17 / 312.0 MB,
+`f32` 176 / 25.8 MB, `f16` 37 / 11.4 MB.
+
+Which tensors carry which quant is the load-bearing portability fact:
+
+- `q6_K` (17): `talker.text_embd.weight [2048, 151936]` (255.3 MB), `talker.codec_embd.weight
+  [2048, 3072]`, and the 15 `code_pred.codec_embd.<g>.weight [2048, 2048]` embedding tables.
+  These are the llama.cpp `Q4_K_M` "embedding/output rows go to Q6_K" rule, applied to
+  embedding tables rather than to an `lm_head`.
+- `q4_K` (250): every attention/FFN projection of the talker and of the code predictor.
+  Distinct matmul shapes: `[2048,2048]`×73, `[2048,1024]`×62, `[2048,6144]`×56,
+  `[6144,2048]`×28, `[1024,2048]`×20, `[1024,3072]`×10, `[1024,1024]`×10, `[3072,1024]`×5,
+  `[2048,3072]`×2, `[2048,151936]`×1. Every one is 2-D and ≥ 1024 on both axes — good
+  GEMM shapes, nothing exotic.
+- `f32` (176) / `f16` (37): all of `spk_enc` (conv weights F16 because K=1/3/5 rows are not
+  block-alignable, per the converter's `should_quantize` policy in `tools/quantize.cpp` +
+  `quantize.sh`; every bias/norm/LayerScale vector F32), plus the talker/predictor norms and
+  the `text_proj` biases.
+
+Metadata the runtime actually reads (talker file): `qwen3-tts.num_code_groups=16`,
+`talker.vocab_size=3072`, `text_vocab_size=151936`, `text_hidden_size=2048`,
+`position_id_per_seconds=13`, `rope.mrope_section=[24,20,20]`, `mrope_interleaved=1`,
+the 7 codec special ids (pad 2148 / bos 2149 / eos 2150 / think 2154 / nothink 2155 /
+think_bos 2156 / think_eos 2157), `language_names` (10: chinese, english, german, italian,
+portuguese, spanish, japanese, korean, french, russian) with their codec ids, the text-side
+special ids (im_start 151644, im_end 151645, tts_pad 151671, tts_bos 151672, tts_eos 151673),
+the whole `generation_config.json` (do_sample, top_k 50, top_p 1.0, temperature 0.9,
+repetition_penalty 1.05, `subtalker_*` mirrors, max_new_tokens 8192), and an embedded
+**Qwen2 BPE tokenizer** (`tokenizer.ggml.model=gpt2`, 151,676 tokens, 151,291 merges,
+eos 151643) — that BPE block is 5.95 MB of the header, i.e. 49 % of the header is vocabulary.
+
+Two live-file details that contradict the upstream doc text (worth knowing when reading
+`docs/ARCHITECTURE.md`: the doc example block is stale against the shipped file):
+`mrope_interleaved` is **1** (doc shows `false`), and `spk_enc.sample_rate` is **24000**
+(doc says 16000). Doc also says "404 tensors on the 1.7B voice_design build" — the base
+build shipped here has 480 because it carries `spk_enc.*`.
+
+### 1.2 Tokenizer/codec file (398 tensors)
+
+One neural audio codec, 12.5 frames/s over 24 kHz mono, hop 1920, 16 codebooks × 2048
+entries (1 semantic + 15 acoustic). Dtype split: `f16` 50 / 143.6 MB, `q8_0` 110 / 76.9 MB,
+`f32` 238 / 70.7 MB. `tok_dec.*` 253 tensors / 201.3 MB, `tok_enc.*` 145 / 89.9 MB.
+
+Composition (metadata + tensor census, both files agree):
+
+- **Encoder** (used only to make voice references, not on the synthesis hot path): SEANet
+  conv stack (init k=7, 4 stages with ratios 8/6/5/4 → cumulative 960×, 64→512 ch, final
+  k=3), an 8-layer transformer (hidden 512, 8 heads, head_dim 64, FFN 2048, RoPE 1e4,
+  LayerNorm+bias, GELU, LayerScale), a downsample conv k=4 s=2 (25 → 12.5 Hz), then the RVQ
+  argmin loop. Metadata records `encoder.num_quantizers=32` with
+  `encoder_valid_num_quantizers=16` — 16 codebooks are used, 16 are dead weight carried from
+  the upstream config.
+- **Decoder** (the hot path: every generated frame): 16 codebook lookups + per-split
+  `output_proj` (256→512), `conv_pre`, an 8-layer transformer (hidden 512, 16/16 heads,
+  head_dim 64, FFN 1024, **sliding window 72**, RMSNorm, SwiGLU, LayerScale, input_proj
+  1024→512 / output_proj 512→1024), a ConvNeXt upampler (`upsampling_ratios=[2,2]`, causal
+  transposed convs + depthwise k=7), then a DAC-style decoder `dec.0..3`
+  (`upsample_rates=[8,5,4,3]`, channels 1536→768→384→192→96, `decoder_dim=1536`) whose blocks
+  are SnakeBeta → causal transposed conv (k = 2·stride) → 3 residual units (dilations 1/3/9)
+  → snake_post → conv_post k=7 → 96→1.
+- All 32 codebook tensors are **F32, 67.1 MB total** (16 encode + 16 decode, `[256, 2048]`),
+  deliberately unquantized: the converter's `should_quantize` keeps RVQ codebooks and their
+  wrapping projections at F32 because nearest-neighbour lookup is sensitive to per-row
+  quantisation noise (`docs/ARCHITECTURE.md` "Quantisation policy"; mirrored in
+  `quantize.sh`). 49 of the codec's 55 3-D conv tensors are **F16** with non-alignable K
+  (7, 3, 2, 1, 16, 10, …) — they cannot be block-quantized at all.
+- **No flow matching, no diffusion, no vocoder-with-ODE.** It is a pure RVQ codec
+  (Mimi/SoundStream/Mimi-style: SEANet + transformer + DAC decoder). That matters for the
+  NPU question: every component is convolution, attention, or embedding lookup.
+
+### 1.3 Where the payload came from, and what the official weights look like
+
+Official upstream (`huggingface.co/api/models`, retrieved 2026-10-19):
+
+| repo | last modified | license | weights |
+|---|---|---|---|
+| `Qwen/Qwen3-TTS-12Hz-1.7B-Base` | 2026-01-23 | apache-2.0 | `model.safetensors` BF16, 1,928,677,440 params, plus an embedded `speech_tokenizer/model.safetensors` |
+| `Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice` | 2026-01-29 | apache-2.0 | same layout, named-speaker tables |
+| `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | 2026-01-29 | apache-2.0 | 1.7B only |
+| `Qwen/Qwen3-TTS-12Hz-0.6B-Base` / `-0.6B-CustomVoice` | 2026-01-29 | apache-2.0 | narrow variant (hidden 1024) |
+| `Qwen/Qwen3-TTS-Tokenizer-12Hz` | 2026-01-29 | apache-2.0 | codec alone, F32, 170,557,441 params |
+
+So the GGUF is **not** the only public form: the safetensors are the canonical form
+(Apache-2.0, no separate community license risk), and the GGUF is a lossy re-quantization of
+them produced by the same author who wrote the runtime (`Serveurperso`), via
+`checkpoints.sh` → `convert.py` (F32 GGUF) → `quantize.sh` (BF16/Q8_0/Q4_K_M).
+Sizes to remember for any re-export: the talker container is 1.93 B params in BF16
+(that count includes the embedded codec), the codec alone is 170.6 M params in F32.
+
+**Portability consequence #1 — the GGUF is a dead end off-ggml.** Q4_K/Q6_K are ggml
+block formats (256-element super-blocks, F16 sub-block scales, 144 B / 210 B per block). No
+ONNX graph, no Vitis-AI/AIE-ML compiler, and no vendor NPU runtime consumes them; AMD's own
+quantizer works from ONNX (`ryzenai-quantize`, QDQ/QOP power-of-two — Ryzen AI Software
+1.8.0 release notes, https://ryzenai.docs.amd.com/en/latest/relnotes.html). Moving the
+payload to a non-ggml stack therefore means starting from the safetensors again, not
+converting the GGUF.
+
+**Portability consequence #2 — the quantization split is semantic, not cosmetic.** The
+converter keeps RVQ codebooks, `input_proj`/`output_proj` around them, and every 1-D and
+non-alignable-conv tensor out of the k-quants precisely because quantizing them breaks voice
+fidelity. Any re-quantization path (GPTQ/AWQ/QDQ-INT8/INT4 for an NPU compiler) inherits that
+constraint: the talker backbone is requantizable, the codec's codebook lookups and SnakeBeta
+convs are not, at least not without re-measuring voice quality.
+
+## 2. What the runtime is
+
+`qwentts.cpp` @ `5916bf7c` (2026-10-16, MIT; upstream model Apache-2.0) is a C++17 ggml
+application — 43 files under `src/` + `tools/`, three binaries (`qwen-tts`, `qwen-codec`,
+`tts-server`) over one public plain-C99 ABI (`src/qwen.h`, `qt_*`, `QT_ABI_VERSION`
+range-checked; `tests/abi-c.c` is compiled `-std=c99 -Wall -Werror -pedantic` on every
+build). It is not a llama.cpp derivative: it links ggml directly and builds its own graphs.
+
+Five graph families, and their shapes are the whole portability story:
+
+| graph | batch/time axis | notes |
+|---|---|---|
+| talker prefill | T = prompt length (hundreds–thousands) | summed text+codec embedding streams, 28-layer GQA decoder, KV cache |
+| talker decode | **T = 1 per audio frame** | 12.5 frames/s → 80 ms of audio per step; `codec_head` → 3072 logits |
+| code predictor (MTP) | **T = 1, 16 steps per frame** (2 prefill + 14 decode, cache reset every frame) | 5-layer decoder, 15 heads × 2048 |
+| speaker encoder | one pass per voice registration | ECAPA-TDNN with mel fused into the graph |
+| codec decode | offline: T = 300-frame chunks; streaming: **T = 1 frame** with persistent conv/KV-ring state | the streaming graph is built once with fixed topology so the CUDA graph cache stays in pure replay |
+
+Per generated frame the host therefore runs ~1 + 16 + 1 tiny graph computes for 80 ms of
+audio, and `tts-server` serializes synthesis FIFO (`--max-batch` defaults to 1, and the
+fleet runs it at 1). This is the archetypal latency-bound, bandwidth-bound,
+batch-1 workload; the only big-M matmuls in the whole system are the prefill and the
+chunked (buffered) codec decode.
+
+Op contract actually issued to ggml (`src/`, counted): `mul_mat` 77, `cont` 70, `add` 57,
+`mul` 40, `rms_norm` 22, `clamp` 15, `rope_ext` 12, `get_rows` 12, `set_rows` 8,
+`scale_bias` 6, `swiglu_split` 5, `sqr` 5, `mean` 5, `elu` 4, `col2im_1d` 4, `soft_max` 3,
+`conv_1d` 3, `im2col` 2, `cumsum` 2, `sin`, `sigmoid`, `tanh`, `div`, `argsort`, `transpose`,
+`permute`, `reshape/view`, plus the CPU-side Philox multinomial sampler.
+
+**No custom ggml op is required today.** The repo doc still says the fork "provides the two
+custom ops the codec needs: `GGML_OP_SNAKE` and `GGML_OP_COL2IM_1D`"; that is stale at the
+pinned commit. Verified 2026-10-19:
+
+- the submodule `ServeurpersoCom/ggml` @ `cc5df49` is upstream `ggml-org/ggml` master
+  (`a05c992`, 2026-10-15) **+2 commits, 0 behind** (GitHub compare API), and
+  `include/ggml.h` is byte-identical to upstream master (`diff` empty);
+- the 2 commits are `193e339` (2026-06-19) "cpu: add fused snake activation
+  (mul -> sin -> sqr -> mul -> add)" — a **CPU subgraph fusion**, not an op (the runtime
+  builds snake with `ggml_sin`/`ggml_sqr`/`ggml_mul`/`ggml_add` in `src/dac-decoder-v2.h:222`),
+  and `cc5df49` "sycl: fix get_rows on reordered quantized weights";
+- `ggml_col2im_1d` is an **upstream ggml API** (same declaration, line 2097, in both trees);
+  the Vulkan backend fuses the snake chain through ggml's generic subgraph fuser
+  (`ggml-vulkan.cpp:14548 ggml_vk_can_fuse_snake`, `vulkan-shaders/snake.comp`).
+
+Consequence: any stock ggml build (upstream, or llama.cpp's vendored ggml) can execute this
+payload; only the CPU snake fusion is fork-specific, and it is a performance nicety.
+
+Backend/device support: every tool takes `--device <name>` (`CUDA0`, `Vulkan0`, `MTL0`,
+`CPU…`, default best available), resolved through the ggml backend registry; the deployed
+container turns CUDA/HIP/Vulkan/Metal/SYCL/RPC/BLAS all OFF and compiles the native-CPU
+backend with OpenMP, with CPU threads = `hardware_concurrency()/2` (`src/backend.h`).
+The registry in this ggml knows CPU, CUDA, Metal, SYCL, Vulkan, WebGPU, ZDNN, OpenCL,
+**Hexagon**, BLAS, RPC, VirtGPU, **CANN**, ZenDNN, **OpenVINO**, ET — and nothing AMD-NPU
+(no `xdna`/`aie`/`vitis`/`ryzen` string anywhere in the ggml tree).
+
+Voice handling (what the fleet depends on): the registry is **process RAM only**; a restart
+empties it, which is why the unit has a boot warmer. `POST /v1/audio/voices` accepts either
+`{name, ref_text, wav_b64}` (server extracts) or `{name, ref_text, spk_b64, rvq_b64}`
+(pre-extracted latents verbatim); `ref_text` present ⇒ ICL clone mode, absent ⇒ x-vector-only
+mode. `.spk` = the ECAPA x-vector; `.rvq` = 16 codes/frame packed 11 bits LSB-first
+(`src/rvq-file.h`). `wav` and `pcm` `response_format` are different audio at the same seed
+(buffered chunked decode vs stateful streaming decode) — the fleet already encodes that rule.
+
+Parity expectations for any port: the upstream harness matches the PyTorch reference per
+stage (embed cos 0.999988, all 28 hidden taps ≥ 0.999893, logits 0.999824) but explicitly
+reports free-run divergence (`CodesFull exact 4.96 %`, `Audio cos 0.095`) and explains why —
+one argmax tie at FP epsilon diverts the AR trajectory. So a migrated endpoint must be
+accepted by listening + per-stage cosine, never by waveform diff against today's audio.
+
+---
+
+## 3. The three answers Main asked for
+
+### ANSWER A — Qwen3-TTS GGUF → ONNX: no, and it does not matter
+
+There is **no GGUF→ONNX path** and there will not be a useful one: GGUF is a weight
+container for ggml, its Q4_K/Q6_K block layouts have no ONNX consumer, and nothing in the
+GGUF carries the *graph* (the talker/predictor/codec structure lives in C++ code, only the
+op-level structure exists in ggml at runtime). But the GGUF is not the payload's only form —
+official Apache-2.0 safetensors are upstream, and **four independent ports of the full
+pipeline already exist**:
+
+| port | form | what it proves |
+|---|---|---|
+| `xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (HF, updated 2026-03-03, apache-2.0) | FP16+FP32 ONNX, fp16/voice_clone + fp16/voice_design splits: `talker_decode.onnx`, `code_predictor.onnx`, `code_predictor_kv.onnx`, `code_predictor_embed_g0..15.onnx`, `text_embedding.onnx`, `codec_embedding.onnx`, shared `speaker_encoder.onnx`, `speech_tokenizer_encoder.onnx`, `speech_tokenizer_decoder.onnx`, `model_cache.npz` | every component of this architecture exports to ONNX, codec decoder included |
+| `zhangluoyang/qwen3-tts-onnxruntime` | export tooling + Python inference for Base/CustomVoice/VoiceDesign | the export is reproducible from the official checkpoints |
+| `Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO`, `aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03) | OpenVINO IR, **INT8**: `talker`, `code_predictor`, `codec_embedding`, `text_model`, `openvino_speaker_encoder_model`, `speech_tokenizer/speech_decoder` | the talker and predictor INT8-quantize for an NPU-targeting stack |
+| `litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17, apache-2.0) | `qwen3_tts_int8.litertlm` / `_int4.litertlm` (LiteRT-LM `OmniEngine`), legacy split graphs `talker_int4.tflite` (prefill_32/prefill_128/decode signatures, KV 1024, blockwise-32 OCTAV INT4), `mtp_folded_int8.tflite` (16 inner steps × 5 layers folded into one graph, in-graph argmax+gather, GPTQ int8, ~5× faster), `codec_partA` (transformer, fp32) + `codec_partB` (conv upsampler, XNNPACK FORCE_FP16, ~2.5×, "ASR-identical") | an NPU/mobile compiler accepts the whole thing — at the price of graph splitting, static prefill buckets, and folding the per-frame inner loop |
+
+What the ports had to concede, and what any port inherits:
+
+- the generation structure does not fit existing engine loops: LiteRT-LM's Engine "does not
+  support this generation structure yet", so their first shape was **three graphs + a
+  host-side loop**, and the MTP step was "invoked 17× per frame" until they folded it;
+- per-step dispatch cost dominates: even on a Snapdragon 8 Elite Gen 5 they keep the MTP
+  loop, text projection, embedding gathers and the codec on CPU "avoiding 16 synchronous
+  CPU↔GPU round-trips per 80 ms frame", putting only the 28-layer talker on the GPU;
+- RTF on that hardware is **2.57–2.97 (Galaxy S26) and 5.65–8.48 (Pixel 10 Pro)** for the
+  0.6 B model, i.e. 2.6–8× slower than real time. Dankclaw's current CPU engine measures
+  **warm RTF 0.55–0.67 with TTFA 94–112 ms** on the 1.7 B model (fleet trial, 2026-10-18).
+  An NPU port is not "faster by default" for this workload; it has to beat that.
+- quantize the LM (int8/int4: proven), keep the codec's codebook lookups and SnakeBeta convs
+  in fp16/fp32 (the LiteRT split does exactly this).
+
+So: the answer to "can the fleet's payload be exported" is *yes — from the safetensors, not
+from the GGUF*, and the work is a pipeline rewrite (multi-graph host loop, per-frame fold,
+static prefill buckets), not a conversion. That is the same conclusion NeuTTS-Air's
+precompiled-graph route encodes: on AMD's Linux stack the graph is vendor-curated, so the
+fleet chooses the model that AMD already compiled rather than compiling Qwen3-TTS.
+
+### ANSWER B — is there any ggml backend for an AMD NPU? Not upstream; three out-of-tree, none for TTS
+
+Checked against the trees today (2026-10-19), because this was the crux:
+
+- **Nothing in-tree.** `ggml-org/llama.cpp` master (4,237 paths, GitHub tree API) contains no
+  `xdna`/`aie`/`vitis`/`ryzen` file or directory; the ggml backend set is CPU, BLAS, CUDA,
+  HIP, Metal, Vulkan, SYCL, OpenCL, WebGPU, MUSA, RPC, CANN (Ascend), **Hexagon** (Qualcomm),
+  OpenVINO (Intel CPU/GPU/NPU), ZenDNN, ZDNN, VirtGPU, ET. The deployed fork's registry
+  matches. The feature request "Feature Request: XDNA backend" (llama.cpp issue **#21725**,
+  opened 2026-04-10) is still **open**, 34 comments; the older #1499 was closed stale
+  2025-04-07. So Main's premise is confirmed: no ggml XDNA backend exists to flip a flag on.
+- **`Cyronius/ggml-xdna`** — a real ggml backend (`XDNA0`) loaded into *stock* llama.cpp via
+  `GGML_BACKEND_PATH`, no fork. It does bfp16 (8-value blocks sharing an exponent) GEMMs from
+  GGUF weights, and is **prefill-only**: it takes only matmuls in prompt chunks ≥ 1024 tokens
+  (`GGML_XDNA_MIN_BATCH` default 1024, `GGML_XDNA_MIN_MFLOP` 256), attention and every decode
+  token stay on the Vulkan device; pinned to llama.cpp release b10944; tested on Strix Point
+  (HX 370) only — Strix Halo "should pass, but nobody has run it on one yet"; Linux support
+  new and untested on hardware by the author; needs `libxrt-npu2`, `/dev/accel/accel0`,
+  unlimited `memlock`. Best measured prefill gain 1.88× on a 27 B model; models under
+  2,048-wide **lose** (0.77× at width 1,536).
+- **`BrandedTamarasu-glitch/OllamaAMDNPU`** — a llama.cpp fork with `-DGGML_XDNA=ON`,
+  explicitly targeting **Ryzen AI MAX / XDNA2 on Linux** (`/dev/accel/accel0`, XRT 2.21.75+,
+  amdxdna, unlimited memlock). It implements **only `GGML_OP_MUL_MAT`**, dispatched to
+  mlir-aie-compiled `.xclbin`s built per fixed tile shape (e.g. `M=2048 K=2048 N=64`,
+  int8 in / int32 out, per-row weight scales cached at first use), with a second slot for a
+  different K. Its own numbers on a Ryzen AI MAX 385 with Llama-3.1-8B Q4_K_M: NPU prefill
+  12–19 t/s vs **Vulkan iGPU 600–930 t/s**, NPU decode **0.65–1.40 t/s** ("Decode does not
+  use the NPU … M=1 per token, no xclbin covers it"), 0.6 % AIE utilization, and the stated
+  conclusion "**For LLaMA 8B on Strix Halo: use Vulkan exclusively**". Single-maintainer,
+  AI-assisted, self-measured — treat as a proof of concept, but it is the closest published
+  picture of what XDNA2 graph execution costs.
+- Also referenced in the same thread: `Atomic-Germ/OpenFlowLM-Next` (whole-model NPU engine,
+  LLM-only — and slower than the iGPU in the same comparison), `GaTmaNnes/xdna2-`,
+  `hardWorker254/Npu-Embeddings-XDNA1` (adds Whisper ASR ops, NPU1/Linux). None is a TTS path.
+
+Why none of them would carry this workload even if installed: the TTS hot loop is **batch-1
+by construction** (one talker step + 16 predictor steps + one codec frame per 80 ms), which
+sits under every dispatch gate those backends define (≥1024 tokens / ≥32768 M·N·K / tile M
+covering only prefill), and the codec half of the pipeline is conv, col2im, gather and
+elementwise ops — not `MUL_MAT` — so it would fall back to CPU op-by-op. The NPU case for
+this host is therefore *not* "run Qwen3-TTS on the NPU"; it is "run a vendor-compiled AR TTS
+graph on the NPU", which is exactly the NeuTTS-Air spike.
+
+### ANSWER C — would `qwentts.cpp` build for the gfx1151 iGPU? Yes, via **Vulkan**, not HIP
+
+- Upstream ships the build scripts and claims the backends: `buildvulkan.sh` ("AMD/Intel GPU
+  (Vulkan)"), `buildcuda.sh`, `buildsycl.sh`, `buildcpu.sh`, `buildall.sh` (all backends,
+  runtime dynamic loading), and README line 6 "runs on CPU, CUDA, Metal, Vulkan". CMake knobs
+  `GGML_VULKAN` / `GGML_HIP` / `GGML_SYCL` are the same ones the fleet's Containerfile turns
+  OFF, so the engine rebuild is a flag change: `-DGGML_VULKAN=ON` plus
+  `--device Vulkan0` on the same command line the unit runs today.
+- The codec's exotic ops are covered by the Vulkan backend, which is the thing that decides
+  whether a GPU build is coherent: `vulkan-shaders/col2im_1d.comp`, `snake.comp`,
+  `swiglu*.comp`, `cumsum*.comp` exist and `supports_op` handles `GGML_OP_COL2IM_1D` /
+  `CONV_TRANSPOSE_1D` / `CUMSUM`. By contrast `COL2IM_1D` appears in **no** HIP, OpenCL,
+  WebGPU, Hexagon or CANN source — a **HIP** (ROCm) build would shard the ConvNeXt/DAC
+  transposed convs back to the CPU per op. On an AMD iGPU the portable path is RADV/Vulkan.
+- `--device` also accepts a multi-device split (`-dev`-style naming is ggml's), and the
+  runtime's documented FP16 guard (`--clamp-fp16`) exists for sub-Ampere CUDA accumulation;
+  nothing in the tree is NVIDIA-specific.
+- Unverified by me: any vendor statement pairing this fork with gfx1151. Supporting evidence
+  that the hardware/driver combination is viable: this very host already runs a Vulkan
+  endpoint on the gfx1151 iGPU (fleet ground truth, 2026-10-19), and Mesa's Vulkan
+  implementation is officially supported on Radeon Software for Linux
+  (AMD release notes RN-AMDGPU-UNIFIED-LINUX-25.20.3). A gfx1151 Vulkan build of this fork is
+  a build-and-measure spike, not an unsupported port.
+- The real constraint is the one already in the fleet docs: the iGPU belongs to another
+  endpoint, and GPU+NPU concurrency has hung Strix Halo machines. An iGPU TTS build competes
+  with the text endpoints for the same silicon; the NPU route's only genuine advantage is
+  that it does not (the XDNA backend authors make the same argument: "Use NPU when the iGPU
+  is busy … no GPU contention").
+
+---
+
+## 4. What any replacement must reproduce, and what the voices cost
+
+Endpoint contract in production today (from the live unit/env, not the doc):
+`tts-server --model qwen-talker-1.7b-base-Q4_K_M.gguf --codec qwen-tokenizer-12hz-Q8_0.gguf
+--alias frank-tts-engine --host 0.0.0.0 --port 8880 --lang auto --max-batch 1`, no published
+port (the façade on :7772 dials the private bridge name), `GET /health`, `GET /v1/models`,
+`GET|POST /v1/audio/voices`, `DELETE /v1/audio/voices/{name}`, `POST /v1/audio/speech` with
+`response_format` wav|pcm (different audio at equal seed) and optional `seed`,
+`max_new_tokens`, `temperature`, `top_k`, `top_p`, `repetition_penalty`, `subtalker_*`.
+Voice registry is RAM-only, so boot warming is mandatory, and `--alias` keeps the quantization
+choice out of every consumer's config.
+
+Live voice census on Dankclaw (2026-10-19, `/home/dank/voices/`): three packs
+(`narrator1..3`), each carrying `reference.wav` (24 kHz mono crop), `reference16k.wav`,
+`manifest.json` (keys: name, kind, generated_utc, pipeline, source, forbidden_region_guard,
+silence_scan, crop, loudnorm, audio, acceptance, acceptance_passed) and ffmpeg analysis
+artifacts. **No `.spk` and no `.rvq` exist on the live host** — the repo's
+`build-voice-pack.sh` treats them as required-but-they-are-not-there, so today the engine
+re-extracts the x-vector (and the ICL codes when a transcript is supplied) from the WAV at
+registration time.
+
+That is the portable part, and it is the same shape AMD's NeuTTS flow wants
+(`encode_reference(reference.wav)` + a reference transcript): the fleet's voice identity is
+*the WAV plus its transcript*, processed at request/registration time by the model's own
+speaker encoder + codec — **never a trained per-voice artifact**. Qwen3-TTS cloning is
+inference-time conditioning (`generate_voice_clone(ref_audio=…, ref_text=…, x_vector_only_mode=…)`;
+`create_voice_clone_prompt` exists purely to avoid recomputing prompt features). So any
+replacement that consumes a WAV (+transcript) keeps all three narrators alive with no
+re-derivation, and if it wants precomputed latents instead, the LiteRT precedent shows what
+that looks like: a 4 KB x-vector `.npy` per voice, extracted offline
+(`extract_speaker_embedding.py`) — for the 1.7 B base checkpoint that vector is 2048-dim
+(`qwen3-tts.spk_enc.embedding_length = 2048` in the shipped GGUF; the LiteRT 0.6 B bundle
+ships a 1024-dim one).
+
+One trap for a mixed-runtime future: `.spk`/`.rvq` files, if they are ever written, are
+engine-specific encodings (ECAPA x-vector layout; 11-bit-packed RVQ stream) and are **not**
+interchangeable between runtimes. Keep `reference.wav` as the pack's source of truth and
+regenerate anything derived per engine — which is what the NeuTTS re-clone plan does anyway.
+
+## 5. Sources
+
+Runtime/payload (all read locally at pinned versions, 2026-10-19):
+
+- `github.com/ServeurpersoCom/qwentts.cpp` @ `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4`
+  (commit date 2026-10-16; also upstream HEAD today): `README.md`, `docs/ARCHITECTURE.md`
+  (860 lines), `src/*`, `tools/*`, `docs/DOCKER.md`, `buildvulkan.sh`/`buildcpu.sh`/`buildall.sh`.
+- submodule `ServeurpersoCom/ggml` @ `cc5df49a6b878db2b68b9a7b45976e279594627d` (2026-10-16);
+  delta vs `ggml-org/ggml` master `a05c992` (2026-10-15) = ahead 2 / behind 0, per
+  https://github.com/ServeurpersoCom/ggml/compare/ggml-org:master...HEAD ;
+  `include/ggml.h` identical to upstream
+  https://raw.githubusercontent.com/ggml-org/ggml/master/include/ggml.h .
+- Live payload bytes: `/home/dank/models/Qwen3-TTS-GGUF/qwen-talker-1.7b-base-Q4_K_M.gguf`
+  (1,219,245,248 B) and `qwen-tokenizer-12hz-Q8_0.gguf` (291,150,624 B), headers parsed from
+  the host; sizes/sha256 match `scripts/download-frank-tts-engine-qwentts.sh`.
+- Live units/env/packs: `dankclaw/voiceservice/{Containerfile.engine,tts-engine-start.sh,tts-engine-warm.sh,quadlet/*}`,
+  `scripts/frank-tts-engine.env`, `scripts/frank-tts.env`, `/home/dank/voices/*`.
+
+Model provenance:
+
+- `https://huggingface.co/api/models/Qwen/Qwen3-TTS-12Hz-1.7B-Base` (+ CustomVoice,
+  VoiceDesign, 0.6B variants, `Qwen3-TTS-Tokenizer-12Hz`), apache-2.0, last-modified
+  2026-01-23 / 2026-01-29; safetensors param counts 1,928,677,440 (BF16, incl. embedded
+  `speech_tokenizer/`) and 170,557,441 (F32 codec).
+- `https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base/raw/main/README.md` (clone API:
+  `generate_voice_clone(ref_audio, ref_text, x_vector_only_mode)`,
+  `create_voice_clone_prompt`; reference stack = PyTorch + `device_map="cuda:0"` + bf16 +
+  flash-attn-2, Python 3.12 recommended).
+- `https://github.com/QwenLM/Qwen3-TTS`.
+
+Portability precedents:
+
+- `https://huggingface.co/xkos/Qwen3-TTS-12Hz-1.7B-ONNX` (2026-03-03) ·
+  `https://github.com/zhangluoyang/qwen3-tts-onnxruntime` ·
+  `https://huggingface.co/Echo9Zulu/Qwen3-TTS-12Hz-CustomVoice-1.7B-INT8-OpenVINO` (2026-03-22) ·
+  `https://huggingface.co/aurora2035/Qwen3-TTS-12Hz-0.6B-Base-OpenVINO-INT8` (2026-03-18) ·
+  `https://huggingface.co/litert-community/Qwen3-TTS-12Hz-0.6B-Base` (2026-10-17) incl.
+  `legacy/README.md` (graph splits, MTP fold, codec partA/partB, RTF table).
+
+AMD NPU / ggml backend status:
+
+- `https://api.github.com/repos/ggml-org/llama.cpp/git/trees/master?recursive=1` (no
+  XDNA/AIE/Vitis backend, 2026-10-19) ·
+  `https://github.com/ggml-org/llama.cpp/issues/21725` (open since 2026-04-10) ·
+  `https://github.com/ggml-org/llama.cpp/issues/1499` (closed stale 2025-04-07).
+- `https://github.com/Cyronius/ggml-xdna` (prefill-only, gates, Linux requirements, results
+  measured 2026-10-05/06 on HX 370) ·
+  `https://github.com/BrandedTamarasu-glitch/OllamaAMDNPU` (XDNA2 ggml backend on Strix Halo,
+  phases 1–12, "use Vulkan exclusively", measured on Ryzen AI MAX 385).
+- `https://github.com/amd/RyzenAI-SW` (main, 669 paths, 2026-08-18): audio content is ASR
+  only (`Demos/ASR/Whisper`, `Demos/ASR/Parakeet-TDT`, `Transformer-examples/ASR/Whisper-AI`);
+  **no TTS example anywhere in the official Ryzen AI Software tree**.
+- `https://ryzenai.docs.amd.com/en/latest/relnotes.html` (Ryzen AI SW 1.8.0: ONNX-format
+  models, ONNX Runtime as the loader, power-of-two QDQ/QOP quantization).
+- Mesa/RADV on AMD iGPUs: `https://www.amd.com/en/resources/support-articles/release-notes/RN-AMDGPU-UNIFIED-LINUX-25-20.3.html`
+  (Mesa Vulkan driver officially supported).
+
+Status: COMPLETE for the assigned slice (payload architecture, runtime architecture,
+portability). `[INFERENCE]` marks: none above except the Mesa-version caveat in ANSWER C,
+which is flagged there.

report/tts-npu-spike-verdict-2026-10-19.md

@@ -0,0 +1,180 @@
+# TTS on the Strix Halo NPU — spike verdict (2026-10-19)
+
+Question: **how can the fleet's TTS engine run on hardware rather than CPU?** Answer in
+one line: not on the NPU — the loader, not the silicon, refuses it; the door that exists is
+the gfx1151 iGPU behind a build flag we already own.
+
+Spike evidence lives in `report/npu-tts-runtime-landscape-2026-10-19.md` (which runtimes can
+reach XDNA2 at all) and `report/qwen3-tts-payload-portability-2026-10-19.md` (what the
+payload is and where it can be moved). This page owns only the **measured spike result** and
+the **model-side contract** neither slice has. Throwaway work ran in containers under
+`/home/dank/spikes/npu-tts` on Dankclaw; nothing in the fleet was changed — `frank-tts` and
+`frank-tts-engine` still serve `narrator1|2|3` on CPU.
+
+Legend: **[host]** live census 2026-10-19 · **[spike]** measured in the spike containers ·
+**[vendor]** primary vendor source, with the version it holds for.
+
+## 1. Verdict
+
+**NO-GO on the NPU, and the wall is a loader.** The XDNA2 device is healthy by AMD's own
+instruments; the runtime that owns this model family opens the device and then declines the
+graph, silently. One run loaded successfully; byte-identical runs never reproduced it, and a
+path that cannot be reproduced is not a foundation. **[spike]**
+
+**Go on the iGPU, and the lever is in this repo.**
+`dankclaw/voiceservice/Containerfile.engine` builds `qwentts.cpp` (commit `5916bf7`) with
+`-DGGML_VULKAN=OFF` written out deliberately beside CUDA/HIP/METAL/SYCL/RPC — the running
+container carries `libggml-base.so.0.27.0` + `libggml-cpu.so.0.27.0` + `tts-server` and
+nothing else. So: `-DGGML_VULKAN=ON` + `--device Vulkan0`, plus `/dev/dri` passthrough in
+the engine quadlet (the ComfyUI pattern: `AddDevice` + `GroupAdd=keep-groups` + a
+world-writable udev rule). The target shape is **co-tenancy** — the engine resident on the
+gfx1151 *beside* `frank-dragoon`, not a handover. §5 carries the census and both gates:
+the **heap gate is PASSED** — RADV's `DEVICE_LOCAL` heap is 77.33 GiB with 65.16 GiB of
+budget while dragoon is loaded, so memory is not the constraint — which leaves the
+contention gate (dragoon's TTFT with a synthesis in flight) and one unproven piece:
+whether qwentts.cpp's codec ops have Vulkan kernels for every op. **[host]** **[spike]**
+
+Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs
+only **12.5 codes/s** (§2), so this is latency headroom and freed cores, not reviving a
+dead service. The independent Strix Halo measurement in the portability slice ("use Vulkan
+exclusively"; NPU decode under 1.5 t/s for an 8B) points the same way.
+
+## 2. The two arms do not share a real-time bar
+
+|Arm|Talker|Codec|Codes per second of audio|Bar|
+|---|---|---|---|---|
+|**Deployed** (`frank-tts-engine`)|`qwen-talker-1.7b-base-Q4_K_M.gguf`|`qwen-tokenizer-12hz-Q8_0.gguf`|**12.5**|~13 tok/s **[host]**|
+|**Spike** (`amd/neutts-air_rai_1.8.0_npu_4K`)|NuTTS-Air, 24-layer Qwen2|NeuCodec `24000 / hop 480`|**50.00**|~50 tok/s **[spike]**|
+
+Measured from the staged NPU artifact — the contract any future arm must satisfy:
+
+|Fact|Value|
+|---|---|
+Vocab|`genai_config.json` `vocab_size` **217,652**; base BPE 151,643; specials 151,644–151,670 **[spike]**|
+Speech tokens|**65,536** `<|speech_N|>` entries, ids **151,671–217,206** contiguous; the vendor template maps a codec index straight to its id, `"".join(f"<|speech_{i}|>")`. A 10,001-code assumption silently truncates the space — encoded references really do reach the 60k range. **[spike]**|
+Stop token|`<|SPEECH_GENERATION_END|>` = 151,670, but the export's `model.eos_token_id` is `[151645, 151643]` (the *chat turn* stop) and `search.eos_token_ids` is unset — pass 151,670 explicitly or speech ends after one token **[spike]**|
+Input ceiling|4096 tokens **total**, input + output **[vendor]**|
+Frame rate|50.00 codes/s confirmed across three references (49.96 / 49.99 / 50.00) **[spike]**|
+Reference encode|`librosa.load(sr=16000, mono=True)` → `[1,1,T]` float → `codec.encode_code(audio_or_path=…)` (vendor `NeuTTS.encode_reference`); CPU RTF 0.28–0.38 on 8 threads, so codes are a cacheable one-time cost, never a per-request charge. `neucodec` also needs `torchtune`; install both `--no-deps --target <dir>` so a second torch never shadows the venv's. **[spike]**|
+Pre-encoded voices|`/spike/logs/codes-narrator{1,2,3}.npy` — 994 / 1,249 / 1,250 codes for 19.90 / 24.98 / 25.00 s **[spike]**|
+
+**Gap that any arm inherits:** the voice packs carry no transcript. `manifest.json` holds
+provenance and crop geometry (`crop.source_speech_run` 98.739–118.636 s of a 22,569 s
+audiobook) and no text, and Dankclaw has no ASR at all — `whisper`, `whisper.cpp`,
+`faster-whisper` all absent. NuTTS conditioning needs `ref_text`; do not invent it. **[host]**
+
+## 3. Exoneration chain — why the blame is not the hardware
+
+Every row is an instrument, not an inference.
+
+|Probe|Result|
+|---|---|
+`xrt-smi examine`|`[0000:c6:00.1] RyzenAI-npu5 aie2p 6x8`, XRT 2.25.37 (vendor extract), NPU fw 1.1.2.65, `amdxdna 7.0.0-38-generic` **[spike]**|
+`xrt-smi validate`|**PASSED**, including `throughput` **[spike]**|
+`pyxrt.device(0)` in-container|constructs — XRT enumerates the device **[spike]**|
+`strace -f -e trace=openat` on the failing load|`genai_config.json` read → `model.onnx` opened → `libxrt_driver_xdna.so.2` loaded → `/dev/accel/accel0` opened `O_RDWR` → GEM ioctls succeed → **EP declines, before touching `dd_metastate_*` or `cache/txn_bins.zip`** **[spike]**|
+`dmesg`, `fuser /dev/accel/accel0`|silent since boot, no holder — the decline never reaches the kernel **[spike]**|
+Hardlinked mirror dirs of the model tree (`rundir*`)|retired, no change — the "metastate shadowing" theory is dead **[spike]**|
+Every logging lever in that build: `GENAI_ORT_VERBOSE_LOGGING`, `GENAI_LOG_ORT_LIB`, `set_log_options`, `set_log_callback(callable)`, `RYZENAI_EP_TRACING`, `RYZENAI_LLM_PROFILE_INIT`|**zero lines** — the decline path is silent **[spike]**|
+Vendor `quicktest`|`Session successfully initialized` — proves the XRT/driver/device path, **not** NPU residency: that build falls back to CPU silently. Prove residency with `/sys/class/accel/accel0/device/mem_usage` during generation. **[spike] [vendor]**|
+
+No NPU token rate was ever measured, so none is quoted here. The single successful load
+(13:17, `[load] 0.84s`, finite logits, XRT banner at *generate* time) has not been
+reproduced by the byte-identical command since, with the model files pristine by inode and
+mtime. **[spike]**
+
+## 4. Host gates the spike proved (so they are not re-litigated)
+
+- **Rootless containers cannot read `/dev/accel/accel0`** (`root:render` 0660): the host
+  gid never enters the userns map. `--group-add keep-groups`, `--userns=keep-id`,
+  `--userns=keep-id:gid=990`, `--group-add render`, `--group-add gid:990` — all fail.
+  Doors: rootful podman (separate storage), or a world-writable udev rule, which the fleet
+  already does for `kfd`/`renderD*`. **[spike]**
+- **Never `dpkg -i` the vendor XRT deb** — it installs a DKMS `amdxdna` over the distro
+  driver. `dpkg-deb -x` the debs and bind-mount the tree at `/opt/xilinx/xrt`; the plugin
+  deb is what carries `libxrt_driver_xdna.so`. **[spike]**
+- In-tree `amdxdna 0.7.0-38` + `iommu=pt` is enough for context creation; a missing device
+  node is *not* the modern failure mode. **[host]**
+- `xrt-smi` inside a container needs `libboost-filesystem1.83.0` +
+  `libboost-program-options1.83.0` (**hyphenated** Ubuntu names, after `apt update`);
+  boost is needed by `xrt-smi` only, never by the OGA/EP path. **[spike]**
+- Headroom is tight and it is host RAM, not a separate pool — see §5 for tonight's census.
+  A multi-GB NPU partition plus an AIE compile workspace does not fit beside
+  `frank-dragoon`. Benign-looking `Failed to alloc dev bo memory, ret -28` is that
+  pressure. **[host]**
+
+## 5. Target shape: the TTS engine shares the gfx1151 with `frank-dragoon`
+
+The deployment Frank wants is **both resident at once**, not a handover. Measured 2026-10-19
+(read-only, nothing loaded or unloaded):
+
+|Measure|Value|Reading|
+|---|---|---|
+iGPU GTT in use|**18,523 MiB of 116,733 MiB**|≈96 GiB of aperture free; this is the real budget (fleet law) **[host]**|
+DRM "VRAM" carve-out|2,048 MiB total, 174 MiB used|what KFD/DRM calls dedicated — **not** the budget **[host]**|
+Host RAM|121 GiB total, 17 GiB available, 79 GiB buff/cache|**GTT pages are system pages**: an iGPU allocation is host RAM consumed, so the two tenants share one pool **[host]**|
+TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that same pool **[host]**|
+
+**Two gates, in this order.**
+
+1. **Heap gate — PASSED.** RADV does **not** hand a container tenant the 2 GiB DRM
+   carve-out. `vulkaninfo` inside the engine's own base image (`ubuntu:24.04` plus
+   `mesa-vulkan-drivers vulkan-tools`, `--device /dev/dri`, rootless) sees
+   `Radeon 8060S Graphics (RADV GFX1151)`, driver `radv`, API 1.4.318, and reports:
+
+   |Heap|size|budget with dragoon loaded|flags|
+   |---|---|---|---|
+   |0|38.67 GiB|32.58 GiB|host-visible, not device-local|
+   |1|**77.33 GiB**|**65.16 GiB**|`DEVICE_LOCAL`|
+
+   The heaps sum to the ~116 GiB aperture in Mesa's ⅓/⅔ APU split — the same RAM twice:
+   a Vulkan allocation is a GTT mapping, so a budget is a ceiling on mappings, not a pile
+   of free silicon memory. ggml's Vulkan backend asks for `DEVICE_LOCAL`, which is heap 1,
+   and the TTS engine needs roughly 4 GiB of it. **Memory stops being the question.** **[spike]**
+   Probe traps: `vulkaninfo --batch` is not a flag in this `vulkan-tools` (it prints the
+   help page and exits 1 — use `vulkaninfo --text`), and the container has **no NVIDIA
+   ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by
+   device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the
+   3090 slot.
+2. **Contention gate.** `frank-dragoon` is the endpoint the operator session runs on, so the
+   honest measurement with both loaded is dragoon's own TTFT and decode with a synthesis in
+   flight — one request each, no ladder. Expect compute-unit sharing rather than memory
+   pressure: the aperture has room, the shaders do not.
+
+**Change surface** — the whole edit, so nobody re-derives it:
+
+1. `Containerfile.engine` build stage: add `libvulkan-dev glslang-dev`. It installs only
+   `ca-certificates cmake g++ git make` today, and ggml's Vulkan backend compiles its
+   shaders at build time — the flag alone fails at configure.
+2. `-DGGML_VULKAN=OFF` → `ON` in the same cmake block; the flag list is written out
+   explicitly, so this is a one-word diff.
+3. Runtime stage: add `mesa-vulkan-drivers`. Without a Vulkan **ICD inside the image** the
+   device node buys nothing — there is no `radeon_icd.json` in the container today.
+4. `frank-tts-engine.container`: `AddDevice=/dev/dri` + `GroupAdd=keep-groups` — **not**
+   `renderD128`. On this hybrid box that node is the NVIDIA card; the iGPU is
+   `renderD129` (`readlink -f /sys/class/drm/renderD129/device/driver` → `amdgpu`).
+5. Engine argv: `--device Vulkan0` — qwentts.cpp's tools take `--device <name>` at the
+   pinned commit.
+
+Gate 1 recipe, already run and **PASSED**: `mesa-vulkan-drivers` + `vulkan-tools` in a
+throwaway container off the same base, `--device /dev/dri`, then `vulkaninfo --text` and read
+the `VkPhysicalDeviceMemoryProperties` heap sizes — no model, no load, no compute. The probe
+trap: `vulkaninfo --batch` is not a flag in Ubuntu 24.04's `vulkan-tools` — it prints the help
+page and exits 1, which reads like a driver failure. Every package involved has a candidate on
+the host's apt (`vulkan-tools` 1.4.341, `libvulkan-dev` 1.4.341, `glslang-dev`/`glslang-tools`
+16.2.0); the build stage runs on the digest-pinned Ubuntu 24.04 base, whose versions are older
+but present. Script: `/home/dank/spikes/npu-tts/vk_probe.sh` on Dankclaw.
+
+Constraints that do not relax for a co-tenant: `/dev/dri` passthrough needs the ComfyUI
+pattern (`AddDevice` + `GroupAdd=keep-groups` + the world-writable udev rule) because
+systemd user-unit spawns carry no supplementary groups; the 64 GiB swap at `/swap.img`
+exists to protect co-tenants, and paging a KV pool is worse than the OOM it prevents;
+`pp_dpm_fclk` reads `auto` (unheld), which only becomes a hazard if the NPU ever joins.
+
+## 6. Re-visit condition
+
+Retry the NPU when **a Linux loader claims the graph**: an `onnxruntime-genai-ryzenai`
+build whose RyzenAI adapter takes the NuTTS-Air Full-Fusion graph on `RyzenAI-npu5`, or that
+model appearing in a runtime's own supported catalog with a Linux runner. Then the
+prerequisites are already mapped and priced: the gated NeuCodec encoder for cloned voices,
+the `ref_text` transcripts, the 4096-token prompt budget, and 50 codes/s as the bar.

scripts/frank-archon.env

@@ -47,10 +47,13 @@ FRANK_ARCHON_TP=2
 # MAX_SEQS x (1 + SPEC_TOKENS) below.
 FRANK_ARCHON_TP_WIRE=wht6
 # Decode slots. A slot costs KV, never decode; the pool is sized to this
-# contract and nothing more, so a FOURTH full-height conversation is REFUSED as
+# contract and nothing more. At 1.8.1 the budget resolver caps the cache at
-# too large to fit — never squeezed, paged or truncated. That is Frank's
+# 556,348 fp8 tokens — 2.8 sessions at 200,000 — so a THIRD full-height
-# explicit trade: the freed MiB went to the expert slab.
+# conversation answers 400 while /health and every shorter chat work fine
-FRANK_ARCHON_MAX_SEQS=3
+# (3 x 50k streams run at 77.6 t/s aggregate). Frank's trade: declare the two
+# slots the engine can hold and let the freed ~1.1 GiB go to the expert plane,
+# instead of advertising a third slot that only exists at shorter prompts.
+FRANK_ARCHON_MAX_SEQS=2
 # Served window. The engine treats `kv group tokens >= max-model-len` as a HARD
 # limit, and models.yml contextWindow MUST quote this number (over-length is a
 # refusal, not a truncation). The pool is sized for seqs x maxlen, not one
@@ -63,12 +66,11 @@ FRANK_ARCHON_PLACEMENT=expert_tiered
 # rest, so growing the pool moves experts off the card and shrinking it spills
 # them onto VRAM the cache wanted. Under-sizing it is a boot failure (36,864
 # refused a boot). The ledger's coverage line is the referee, not this file.
-# Sized so the cache clears the 3 x MAX_MODEL_LEN contract: the 1.8.1 boot at
+# Measured, not guessed: 45,568 (PR #30) moved host_pool to 45.09 GiB and left
-# 44,928 resolved the cache to 556,348 tokens — 2.8 sessions — and three
+# vram_kv byte-identical at 4.20 GiB / 556,348 tokens on both ranks — extra
-# concurrent full-height requests came back 400 in 0.0 s. Tokens scale with the
+# host RAM above what the expert plane's footprint needs buys no cache. Back at
-# GiB the cache wins, so +640 MiB a rank buys ~44,000 tokens of ceiling with
+# 44,928, the size that boots clean.
-# ~2.5% off the expert slab.
+FRANK_ARCHON_HOST_POOL_MIB=44928
-FRANK_ARCHON_HOST_POOL_MIB=45568
 # VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
 # floor is built from it.
 FRANK_ARCHON_GPU_HEADROOM_MIB=96
@@ -117,9 +119,8 @@ FRANK_ARCHON_NGRAM_PLACEMENT=ram
 # mid-pin with an anonymous allocator error. ttm-preflight.sh raises the knob
 # itself before refusing by name, so recovery is a boot, not a visit.
 # Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT
-# per the bats ledger test; at the 45,568 pool that computes to 150,200 MiB, so
+# per the bats ledger test; at the 44,928 pool that computes to 148,920 MiB.
-# the floor sits just above it.
+FRANK_ARCHON_TTM_MIN_MIB=150000
-FRANK_ARCHON_TTM_MIN_MIB=152000
 FRANK_ARCHON_TTM_TARGET_MIB=204800
 # Escape hatch for experiments. Unset -> no extra argv; the serve script expands
 # this unquoted on purpose, so keep multi-flag values as one string.

serverAppConfigs-docs/Dankclaw/Frank TTS.md

@@ -111,6 +111,21 @@ own `/health` answers 200 with the backend name and the voice list it resolved
 without dialing the engine, so "façade healthy, engine down" is a real state: it
 shows up as a failed synthesis, never as a down probe.
 
+## Where the engine computes
+
+CPU, and only CPU is compiled in: `Containerfile.engine` builds `qwentts.cpp` with
+`-DGGML_VULKAN=OFF` written out explicitly, so the container carries `libggml-cpu` and no
+other backend. The realistic accelerator is the gfx1151 iGPU behind that flag — plus
+`/dev/dri` passthrough, the ComfyUI pattern — on the same payload and the same pinned
+commit, and the intended shape is **co-tenancy**: the engine resident on the card beside
+`frank-dragoon`, not a handover. Memory is not what decides that; shader contention is, and
+the census plus both gates are in `report/tts-npu-spike-verdict-2026-10-19.md` §5.
+**The Strix Halo NPU is not a door at all**: XDNA2 executes only precompiled AIE
+partition programs, and AMD's own Linux runtime opens `/dev/accel/accel0`, succeeds at the
+ioctls, then declines this model family's graph without emitting a single log line. The
+measured record and the model-side contract live in `report/` at the repo root
+(`tts-npu-spike-verdict-2026-10-19.md`).
+
 ## Files
 
 | Path | What |

serverAppConfigs-docs/LLM/LLM Performance Diagnosis.md

@@ -1,9 +1,9 @@
 ---
 type: reference
 title: LLM Performance Diagnosis
-description: "Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution."
+description: "Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and step-shape attribution."
 tags: [performance, profiling, moe, pcie, prefix-cache, routing, telemetry]
-timestamp: 2026-10-05T00:00:00-07:00
+timestamp: 2026-10-19T00:00:00-07:00
 ---
 
 # LLM Performance Diagnosis
@@ -213,9 +213,75 @@ function directly with synthetic bodies (stdlib-only policy module)
 conversation identity is the FIRST N KiB of prompt — a shorter tested prefix
 hashes differently per turn and pinning silently never happens.
 
+## 9. Decode cadence under a prefill (the step-shape census)
+
+"Decode turns to sludge the moment a long prompt arrives, then recovers" is not a
+link or a gather story — on a chunked-prefill engine the unit of interference is
+**the step**. A step that carries a prefill chunk takes the chunk's time, and every
+decode row sharing that step waits that long for its next token. Chunk size is
+`min(remaining, --checkpoint-interval, --max-num-batched-tokens)`, so on a hybrid
+model the interval caps it before the budget does.
+
+Census it without touching the model — the engine's own counters (`/stats`,
+`/metrics`, cumulative since boot):
+
+```
+decode-only step = step_ns_decode / steps_decode
+shared step      = (step_ns - step_ns_decode) / (steps_prefill + steps_mixed)
+```
+
+Then bucket a day of `decode_tokens_per_second` by `prefill_tokens_per_second`
+(Prometheus `query_range`, step 120 s) and **check the lag**: correlate decode with
+prefill at lag 0 and again at 2-14 minutes. Lag-0-only ⇒ step-time contention, the
+chunk is holding the device. A lagged tail ⇒ residency churn — the prefill's working
+set evicted something decode needs (expert slab, KV) and decode pays after the
+prefill is over. `preemptions` separates the third shape: a preempted-and-recomputed
+sequence stops producing entirely and then re-prefills from token 0, which is a
+stall, not sludge.
+
+Knobs, in the order worth trying them (radiance GUIDE §4.5):
+
+- `--prefill-interleave N` — after a chunk step taken beside decoders, the next N
+  steps carry decode rows only and admit nothing. Decoders keep their cadence; the
+  prompt pays that in TTFT. Counts as `engine_steps_held_total`.
+- `--busy-prefill-chunk MODE` — the chunk's share of a step that has other work:
+  `auto` (default) reserves a row plus the draft window per decode row behind it and
+  an eighth of the budget per waiting prefill/request, `off` lets the chunk take the
+  whole step (admission never runs while it does), `N` pins the share. Never under
+  the prefill stager's smallest pass — the startup line names it, and below it every
+  routed layer reads its pooled experts synchronously, which costs the prefill far
+  more than sharing saves. Counts as `engine_prefill_chunks_capped_total`.
+- `--max-num-batched-tokens` — the blunt rung, and the only one older pins have. The
+  vendor's curve on a 48-layer MoE at 200k on two cards: chunk 512 → 1750 t/s, 1024
+  → 2200, 2048 → 2706 (a 185k prompt: 138 s against 91 s), so halving the stall
+  costs ~19 % of prefill. Raising it past the interval buys nothing and loses expert
+  residency to the arena.
+- `--prefill-span N` — the inverse trade: a *lone* prefill carries N intervals a step
+  while every shared step is planned at a smaller shared budget, which on an
+  expert-tiered box lends that arena back to the expert slab. Vendor TP2
+  expert-tiered TTFT deltas vs a plain 2048 budget: −6.1 % at 2k, +2.3 % at 18k,
+  +49.1 % at 250k — long prompts only.
+
+Two traps. **The arena/slab coupling makes a shared step doubly expensive on an
+expert-tiered shape**: the full-plan arena takes VRAM the resident expert slab would
+have had (320 MiB/card at TP2 for a 4096 budget), so the decode rows inside that step
+also read experts over the link — a smaller shared budget helps twice. And **re-check
+`prefix_hit_rate` / `prefix_evictions` after any chunk change**: the arena and the KV
+pool are the same memory, and the failure looks like turn two taking as long as turn
+one.
+
+Availability is a version question, not a config question: `--busy-prefill-chunk` and
+`--prefill-interleave` are absent from older pins — prove it on the running container
+(`podman exec <c> <binary> --help | grep -iE "prefill|chunk"`), never from the vendored
+tree, and check the source clone's depth before dating a flag from `git log -S`
+(a `--depth 1` clone reports its own tip for every search).
+
+Run on: [[frank-archon]] (radiance 1.0.13, TP2 `expert_tiered`, budget 2048 = interval
+2048) — the measured census and the verdict for this fleet's shape are that page's.
+
 ## Related
 
 -  the window discipline and usage-counter law
 - [[PCIe Link and Atomics]] — link probing and the streaming-vs-compute ceiling
 - [[RDNA4 MoE Expert Offload VRAM Budget]] — the plan behind duty-cycle terms
--  the compile fix for the pointwise-glue bound
+- [[frank-archon]] — the endpoint where §9's step-shape census was run

serverAppConfigs-docs/LLM/frank-archon.md

@@ -239,6 +239,29 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
 - **MNBT 2048.** 2560 collapses decode to 2.6-7.7 t/s (MTP resonance); the w4 file tier streams
   in place only at MNBT ≤ 2048 and the activation arena scales with the chunk (32768 → 33.3
   GiB, refused).
+- **Decode pays for the chunk it shares a step with.** Census of the live engine
+  (radiance 1.0.13, `/stats` + `/metrics`, cumulative over the 2026-10-15 boot):
+  1,435,585 decode-only steps at **18.7 ms** each against 127,991 prefill+mixed steps
+  at **317 ms** — 60 % of engine wall-clock spent inside 8 % of the steps, because the
+  chunk is `min(remaining, checkpoint interval 2048, MNBT 2048)` = 2048 rows and every
+  decode row in that step waits the whole chunk. Day of Prometheus (120 s step, 721
+  samples): `decode_tokens_per_second` median **13.3 t/s** in the 56 % of samples where
+  `prefill_tokens_per_second` ≥ 1000, against 106-240 t/s in the light-prefill buckets;
+  bucketed by mean step time it is 128.7 t/s at 0.02-0.05 s steps and 12.0 t/s at
+  0.15-0.4 s. The stall is **during**, not after — decode in windows 2-14 min after a
+  busy-prefill window sits at the idle baseline (58-64 vs 77-85 t/s, the residual
+  confounded by co-scheduling), so no lingering expert-residency penalty at 2-minute
+  resolution, and `preemptions: 0` rules out preempt-and-re-prefill. Method:
+  [[LLM Performance Diagnosis]] §9.
+- **The knobs for it are not in this pin.** `--busy-prefill-chunk` and
+  `--prefill-interleave` (radiance GUIDE §4.5 — a shared chunk's cap, and N decode-only
+  steps after each chunk) are absent from 1.0.13: proven on the running container
+  (`podman exec frank-archon radiance --help` prints no `prefill`/`chunk` flag), and the
+  2026-10-18 source tree carries both. The only crank this pin has is MNBT, and that is
+  an arm and not an edit: the vendor prices a 1024 chunk at −19 % prefill to halve the
+  stall, and this page's own MNBT law already knows 2560 collapses decode (MTP
+  resonance). An image bump is gated by the 1.2.0 numerics dead end below, so any try
+  re-scores draft acceptance before it is trusted.
 - **Slots cost KV and rows, never speed.** `--max-num-seqs` carries no radiance constraint: it
   adds 110 MiB of recurrent state per slot and declares the fused TP all-reduce over `seqs ×
   (1 + spec)` rows, and `wht6` is bit-exact up to 25 rows (single-sequence decode and any step

serverAppConfigs-docs/LLM/index.md

@@ -35,7 +35,7 @@ the endpoint's **Eras** table, and the numbers a retired era measured are gone.
 | [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. |
 | [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. |
 | [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. |
-| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. |
+| [[LLM Performance Diagnosis]] |  Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and step-shape attribution. |
 | [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. |
 | [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
 | [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |

serverAppConfigs-docs/index.md

@@ -154,7 +154,7 @@ is a locator: one row, one clause, no numbers.
 | [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. |
 | [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. |
 | [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. |
-| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. |
+| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and step-shape attribution. |
 | [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. |
 | [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
 | [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |

tests/frank_archon_unit.bats

@@ -135,14 +135,17 @@ nosudo_bin() {
     grep -q '^FRANK_ARCHON_MAX_MODEL_LEN=200000$' "$ENV_FILE"
     ml=$(awk -F= '/^FRANK_ARCHON_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE")
     [ "$ml" -ge 177356 ]
-    # Concurrency: 3 slots x the served window is the capacity contract, and
+    # Concurrency: 2 slots x the served window is the capacity contract at
-    # the fused TP all-reduce is declared over seqs x (1 + spec) = 9 rows,
+    # 1.8.1. The budget resolver caps the cache at 556,348 fp8 tokens — 2.8
+    # sessions at 200,000 — so three concurrent full-height chats answer 400,
+    # and the shipped CLI has no KV-size flag to raise it (removed at 1.3.0).
+    # The fused TP all-reduce is declared over seqs x (1 + spec) = 6 rows,
     # under the 25-row bound at which the wht6 wire stays bit-exact. The
     # "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
     # (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
     # seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
     # tied inside the +-3% noise floor — slots buy concurrency, not speed.
-    grep -q '^FRANK_ARCHON_MAX_SEQS=3$' "$ENV_FILE"
+    grep -q '^FRANK_ARCHON_MAX_SEQS=2$' "$ENV_FILE"
     grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
     grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
     grep -q '^FRANK_ARCHON_SPEC_TOKENS=2$' "$ENV_FILE"
@@ -155,7 +158,7 @@ nosudo_bin() {
     # so the pool's token count is a boot-ledger line — verified live by the
     # capacity leg of scripts/probe-frank-archon-kv.py at every cutover.
     grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
-    grep -q '^FRANK_ARCHON_HOST_POOL_MIB=45568$' "$ENV_FILE"
+    grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
     grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
     grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
     # Prefix cache: pinned host RAM only (the disk tier was removed), and the