From ea2fc6440b27606aa352e31ee030e053fd8985d0 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 19 Oct 2026 14:37:29 -0700 Subject: [PATCH] docs(frank-tts): record the iGPU cutover, close the NPU question MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cutover PR merged the runtime; this carries the surface that has to be true afterwards. - Engine page: the shape is now `--device Vulkan0` on the gfx1151 as a co-tenant beside frank-dragoon; the registry/thread/streaming laws gain a fourth — the device law — whose numbers stay in the verdict report, and the deploy record gets the cutover steps including the `glslc` configure gate. The CPU-era arm/rollback prose keeps the qualitative argument and drops the dead numbers per the purpose test. - Endpoint page: "Where the engine computes" now says iGPU + co-tenancy, and states the NPU verdict in one paragraph instead of a research tour. - Quadlet comment: it claimed supplementary groups are what grant render access. On this host the udev rule world-writes the render nodes and the container's `id` shows no groups at all, so keep-groups is belt-and-braces; the RSS note now matches the device shape instead of the CPU trial. - Verdict report: gate 2 (contention) recorded as measured after the cutover, and the build-stage prediction corrected — the package is `glslc`, not `libvulkan-dev glslang-dev`. - index.md regenerated (one description changed). --- .../quadlet/frank-tts-engine.container | 29 +++++---- report/tts-npu-spike-verdict-2026-10-19.md | 45 ++++++++----- .../Dankclaw/Frank TTS Engine.md | 63 ++++++++++++------- serverAppConfigs-docs/Dankclaw/Frank TTS.md | 23 +++---- serverAppConfigs-docs/index.md | 2 +- 5 files changed, 98 insertions(+), 64 deletions(-) diff --git a/dankclaw/voiceservice/quadlet/frank-tts-engine.container b/dankclaw/voiceservice/quadlet/frank-tts-engine.container index 752c02b..65c65b4 100644 --- a/dankclaw/voiceservice/quadlet/frank-tts-engine.container +++ b/dankclaw/voiceservice/quadlet/frank-tts-engine.container @@ -40,19 +40,24 @@ # speaks to the server from inside its own namespace, which is where the address # is valid. # -# MEMORY / THE RENDER NODE. Host RSS stays near the 5.7 GiB the CPU trial -# measured (weights + context resident); no memory limit is set because the box -# has the room and a limit would convert a busy moment into an OOM kill -# mid-utterance. +# MEMORY / THE RENDER NODE. Host RSS is ~1.7 GiB, not the size of the GGUFs: +# the Vulkan backend copies both tensors into device memory at load, so they +# stop counting in the process's RSS (measured after the 2026-10-19 cutover — +# `podman stats` / `ps -o rss`). No memory limit is set: the box has the room +# and a limit would convert a busy moment into an OOM kill mid-utterance. # -# `AddDevice=/dev/dri` + `GroupAdd=keep-groups` are what make the iGPU visible, -# and both are required: a rootless container that drops its supplementary -# groups has no render permission, and the Vulkan ICD then enumerates nothing — -# this is the same ride ComfyUI takes (dankclaw/comfyui, plus the udev rule that -# makes the nodes group-writable). The node is NOT named: this host is hybrid, -# renderD128 is the NVIDIA card and renderD129 the amdgpu iGPU, so a node number -# in the unit is a guess that a re-probe can invalidate, while the directory -# hands the container every render node and the driver picks by name. +# `AddDevice=/dev/dri` is what makes the iGPU visible; without it the Vulkan +# ICD enumerates nothing and the server dies on device lookup after it has +# already read both GGUFs. `GroupAdd=keep-groups` is belt-and-braces, not the +# thing that works here: the units podman spawns carry no supplementary groups +# at all, and this host grants render access a different way — the udev rule +# that makes the render nodes world-writable (same rule ComfyUI rides, see +# dankclaw/comfyui). Measured: `id` inside this container lists only the +# container's own group, and `tts-server` still holds renderD129. +# The node is NOT named: this host is hybrid, renderD128 is the NVIDIA card and +# renderD129 the amdgpu iGPU, so a node number in the unit is a guess that a +# re-probe can invalidate, while the directory hands the container every render +# node and the driver picks by name. # # /dev/kfd stays absent on purpose: that is the ROCm compute interface, owned by # the engines that use HIP. A Vulkan endpoint has no use for it, and handing a diff --git a/report/tts-npu-spike-verdict-2026-10-19.md b/report/tts-npu-spike-verdict-2026-10-19.md index 5646585..85f5c8f 100644 --- a/report/tts-npu-spike-verdict-2026-10-19.md +++ b/report/tts-npu-spike-verdict-2026-10-19.md @@ -136,25 +136,36 @@ TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the 3090 slot. -2. **Contention gate.** `frank-dragoon` is the endpoint the operator session runs on, so the - honest measurement with both loaded is dragoon's own TTFT and decode with a synthesis in - flight — one request each, no ladder. Expect compute-unit sharing rather than memory - pressure: the aperture has room, the shaders do not. +2. **Contention gate — PASSED, measured 2026-10-19 after the cutover.** With a 16.9 s + synthesis occupying the device, `frank-dragoon` answered a streamed 124-token request at + **17 ms to first token and ~35 tok/s decode**, against 17 ms and ~33 tok/s for the same + request with the card idle. Compute-unit sharing showed up as nothing. One request per + arm, no ladder: these are the gate's evidence, not a throughput record, and the engine's + own shape lives in [[Frank TTS Engine]]. -**Change surface** — the whole edit, so nobody re-derives it: +**Change surface — as landed** (PR #34, `6248f02`; the CPU shape stays reachable as the +rollback): -1. `Containerfile.engine` build stage: add `libvulkan-dev glslang-dev`. It installs only - `ca-certificates cmake g++ git make` today, and ggml's Vulkan backend compiles its - shaders at build time — the flag alone fails at configure. -2. `-DGGML_VULKAN=OFF` → `ON` in the same cmake block; the flag list is written out - explicitly, so this is a one-word diff. -3. Runtime stage: add `mesa-vulkan-drivers`. Without a Vulkan **ICD inside the image** the - device node buys nothing — there is no `radeon_icd.json` in the container today. -4. `frank-tts-engine.container`: `AddDevice=/dev/dri` + `GroupAdd=keep-groups` — **not** - `renderD128`. On this hybrid box that node is the NVIDIA card; the iGPU is - `renderD129` (`readlink -f /sys/class/drm/renderD129/device/driver` → `amdgpu`). -5. Engine argv: `--device Vulkan0` — qwentts.cpp's tools take `--device ` at the - pinned commit. +1. `Containerfile.engine` build stage: add **`libvulkan-dev glslc spirv-headers`**. The + prediction above (`libvulkan-dev glslang-dev`) is wrong on two counts and the configure + says so: ggml looks for `glslc`, which Ubuntu's `glslang-tools` does **not** ship (it + ships `glslangValidator`), and `find_package(SPIRV-Headers CONFIG REQUIRED)` needs the + distro's `spirv-headers`, which does carry the CMake config. `glslc` is its own source + package on noble, at 2023.8 — old enough that only `GL_KHR_cooperative_matrix` compiles, + so the coopmat2 / integer-dot / bf16 / micro-float shader families are built out. Suspect + that compiler before the device if a Vulkan prefill ever reads as slow. +2. `-DGGML_VULKAN=OFF` → `ON`: one word in the written-out flag list. +3. Runtime stage: `mesa-vulkan-drivers` **and** `libvulkan1` — the ICD and its loader. An + image with a loader and no ICD enumerates zero devices. +4. `frank-tts-engine.container`: `AddDevice=/dev/dri` + `GroupAdd=keep-groups` — the + DIRECTORY, never the node: on this hybrid box `renderD128` is the NVIDIA card and + `renderD129` the amdgpu iGPU (`readlink -f /sys/class/drm/renderD129/device/driver` → + `amdgpu`). On this host the udev rule already makes the nodes world-writable, so the + group is belt-and-braces, not the gate. +5. Engine argv: `--device Vulkan0`, reached through `QWENTTS_DEVICE` in + `scripts/frank-tts-engine.env`. **An empty value passes no flag at all** — that is the + CPU rollback, and it is why the entrypoint's default is an `-v` test rather than `:=`, + which also fires on a set-but-empty value and would silently undo the rollback. Gate 1 recipe, already run and **PASSED**: `mesa-vulkan-drivers` + `vulkan-tools` in a throwaway container off the same base, `--device /dev/dri`, then `vulkaninfo --text` and read diff --git a/serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md b/serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md index 8ff5ad8..bee466a 100644 --- a/serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md +++ b/serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md @@ -1,35 +1,36 @@ --- type: reference -description: "The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes." +description: "The engine behind the fleet's speech endpoint: its Vulkan build on the iGPU, its private bridge, and what it publishes." tags: - dankclaw - tts - voice - quadlet - - cpu -timestamp: 2026-10-18 + - vulkan +timestamp: 2026-10-19 --- # Frank TTS Engine -`frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** in RAM once and synthesizing from there. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page. +`frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** loaded once and synthesizing from there — **on the gfx1151 iGPU** (ggml's Vulkan backend, `--device Vulkan0`) since 2026-10-19, and in host RAM before that. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 when asked, and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page. -**Deployed 2026-10-18.** The chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified, 18 s), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy 31 s after the build, **4.0 GiB** resident on a host that kept 17 GiB available, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user. +**Deployed 2026-10-18, cut over to the iGPU on 2026-10-19.** The first chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy shortly after the build, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user. **Host:** Dankclaw (`192.168.0.143`), rootless Podman, user `dank` — **Port:** 8880, **never published** — **Network:** the private `systemd-frank-tts-engine` bridge (10.89.5.0/24) and no other — **Deployer:** `frank-tts-engine` — **Consumers:** `frank-tts` (7772) and, through it, every voice request in the fleet including [[Open Notebook]]'s podcast renders. -## Facts (verified 2026-10-18 against the pinned upstream commit, the Hub, and the CPU trial on this host) +## Facts (verified 2026-10-18 against the pinned upstream commit, the Hub and the CPU trial; the device rows re-verified on the live iGPU shape 2026-10-19) | Item | Value | |------|-------| | Engine | `tts-server` from qwentts.cpp, pinned at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (`dankclaw/voiceservice/Containerfile.engine`). The source is **not vendored**: the image clones that commit and its recorded `ggml` submodule, so the build needs outbound HTTPS and the `cfhash` label is what keeps an unchanged re-deploy offline | -| Build line | `cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF && cmake --build build -j 8` — the exact line the 2026-10-18 trial ran on this box. `GGML_BLAS=OFF` is **forced**: Dankclaw has `libopenblas.so.0` and no BLAS dev headers, so upstream's own `Dockerfile` and `buildcpu.sh` (both of which pass BLAS on) cannot be copied. No `-DGGML_NATIVE` is passed, and ggml defaults it on, so the binary is `-march=native` for the machine that built it: **build this image on the host that runs it** | -| Image | built locally as `franksdank/frank-tts-engine:latest`, two stages from one digest-pinned base `docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac` (index digest resolved 2026-10-18). Runtime stage: `libgomp1` (OpenMP is on), `curl` (the unit's `HealthCmd` runs as `/bin/sh -c`, and the warmer speaks HTTP), uid **10000**, `LD_LIBRARY_PATH=/app` because ggml builds shared by default and the link-time RPATH names the build tree | +| Build line | `cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=ON -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF && cmake --build build -j 8`. **`GGML_VULKAN=ON` since 2026-10-19**, and that door needs three packages in the build stage: `libvulkan-dev`, `spirv-headers`, and **`glslc`** — which Ubuntu's `glslang-tools` does NOT ship (it ships `glslangValidator`, and `find_package(Vulkan COMPONENTS glslc REQUIRED)` fails without the package). The archive's `glslc` is 2023.8: `GL_KHR_cooperative_matrix` compiles, the coopmat2 / integer-dot / bf16 / micro-float shader families do not. `GGML_BLAS=OFF` is **forced**: Dankclaw has `libopenblas.so.0` and no BLAS dev headers, so upstream's own `Dockerfile` and `buildcpu.sh` (both pass BLAS on) cannot be copied. No `-DGGML_NATIVE` is passed, and ggml defaults it on, so the binary is `-march=native` for the machine that built it: **build this image on the host that runs it** | +| Image | built locally as `franksdank/frank-tts-engine:latest`, two stages from one digest-pinned base `docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac` (index digest resolved 2026-10-18). Runtime stage: `libgomp1` (OpenMP is on), `curl` (the unit's `HealthCmd` runs as `/bin/sh -c`, and the warmer speaks HTTP), `libvulkan1` + `mesa-vulkan-drivers` (the loader and the RADV ICD — a loader with no ICD enumerates ZERO devices, and the server then dies in device lookup after both GGUFs have been read), uid **10000**, `LD_LIBRARY_PATH=/app` because ggml builds shared by default and the link-time RPATH names the build tree. **No NVIDIA ICD on purpose**: exactly one Vulkan device in the enumeration is what makes the name `Vulkan0` safe to write down | | Talker | `qwen-talker-1.7b-base-Q4_K_M.gguf`, **1 219 245 248 B**, sha256 `ea393ebaf2167ea23ce9fc18b093822851358a950d7075cd47ab4f6ce23e887d` (the Hub's LFS pointer oid) | | Tokenizer | `qwen-tokenizer-12hz-Q8_0.gguf`, **291 150 624 B**, sha256 `1883beeed99348fc35e23dd225e9082f93f6f8c109330a33d935baa8acdbfd94` | | Source of both | `https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF`, fetched by `scripts/download-frank-tts-engine-qwentts.sh` into `/home/dank/models/Qwen3-TTS-GGUF`, mounted read-only at `/models`. It has to be the **base** variant: the engine refuses every voice registration unless the model reports type `base`, so on `customvoice`/`voicedesign` the cloned-voice product does not exist. A 0.6 B talker (628 905 056 B) exists and is **not** the decided artifact | -| Memory | **5.7 GiB peak RSS** with both files resident (2026-10-18 trial). No `LimitMEMORY`/`MemoryMax`: a limit would turn a busy moment into an OOM kill in the middle of an utterance | -| Concurrency | `--max-batch 1`: a serialised FIFO. Warm RTF on this box is 0.55–0.67 against real time 1.0, and that margin is the entire headroom a live stream has; two concurrent syntheses split the same cores and the second caller's first byte additionally waits for the first utterance to finish | +| Device | `--device Vulkan0`: the gfx1151 iGPU, **shared with [[frank-dragoon]] as a co-tenant, never a handover**. Proven on the live pair, not inferred: `fuser -v /dev/dri/renderD129` lists `tts-server` beside `flash_serve`. Both gates (the RADV heap and dragoon's contention) are §5 of `report/tts-npu-spike-verdict-2026-10-19.md`, which also carries the change surface. **Rollback: `QWENTTS_DEVICE=` (empty) in `scripts/frank-tts-engine.env`, redeploy this one service** — the entrypoint then passes no `--device` at all, which is the CPU shape, no rebuild, nothing else restarts | +| Memory | **1.7 GiB host RSS** since the iGPU cutover — the weights ride the device. No `LimitMEMORY`/`MemoryMax`: a limit would turn a busy moment into an OOM kill in the middle of an utterance | +| Concurrency | `--max-batch 1`: a serialised FIFO. Warm RTF on the iGPU measures **0.20** (11.04 s of audio in 2.23 s) with `frank-dragoon` live on the same card, so a live stream has room to spare; a second concurrent synthesis would share the device with the first and make the caller behind it wait out the whole first utterance, so the FIFO stays a decision, not an accident | | Network | `systemd-frank-tts-engine` (10.89.5.0/24, `dankclaw/voiceservice/quadlet/frank-tts-engine.network`) and **nothing else**, with **no `PublishPort` at all**. `frank-tts` joins the same bridge and dials `http://frank-tts-engine:8880` | | Voices | `/home/dank/voices` mounted read-only at `/voices`, and it is the **warmer** that reads it, not the server: `tts-server` receives reference audio as base64 in a registration body and keeps its registry in memory only | | Health | `GET /health` → `{"status":"ok"}`, served by the HTTP thread rather than the synthesis queue, so a busy engine still reads healthy. Probed by the unit (`HealthCmd`, 30 s interval, 120 s start period) **inside the container's namespace**, because there is no published port to probe from outside | @@ -37,18 +38,11 @@ timestamp: 2026-10-18 ## Why this engine, and not the alternatives -The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. RTF only has to stay under 1.0 with margin; what separates the arms is first-byte latency and **spread**. Measured on this host the same day (CPU; the full A/B is the section *"Decision: arm A for live output (2026-10-18)"* in , which is the source of these numbers): +The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. On that criterion the resident `tts-server` (arm A) beat the mainline per-request CLI path (arm B) on this box the same day, and the deciding property was **spread**: the resident server repeats the same first-audio latency every run, while the CLI path varies enough that some runs fall behind real time and a live listener hears the stream stall mid-sentence. That A/B was a CPU measurement and the endpoint has since moved to the iGPU, so the shape of the argument is what is kept here; no page carries its numbers any more, and they are not worth re-deriving. -| Measure | Resident `tts-server` (arm A, chosen) | Mainline per-request CLI path (arm B) | -|---|---|---| -| first audio byte | **86–451 ms** end to end; **94–112 ms** server-side | model load, per request | -| warm real-time factor while streaming | **0.55–0.67** | 0.93–0.99 | -| run-to-run spread | **±0.01** | ±0.25 | -| HTTP surface | serves `/v1/audio/speech` itself | none — façade CLI only | +The rollback to the other backend stays in the repo untouched — `TTS_BACKEND=llama-tts-cli` in `scripts/frank-tts.env` still selects the per-request CLI engine, and that file's header documents what it costs. -The spread is the whole argument: at ±0.01 the engine finishes synthesizing 1.5–1.8× faster than the audio plays, every run; at ±0.25 some runs fall behind real time and a live listener hears the stream stall mid-sentence. The rollback path stays in the repo untouched — `TTS_BACKEND=llama-tts-cli` in `scripts/frank-tts.env` still selects the per-request CLI engine, and that file's header documents what it costs. - -## Three laws this service lives under +## Four laws this service lives under **1. The registry law → the unit warms at boot.** The voice registry is **memory only**: a restart forgets every voice, and re-registering one does **not** warm it — only synthesizing does. A boot that registers nothing leaves the façade answering 400 for every name, which is why `frank-tts-engine.container` carries @@ -64,10 +58,12 @@ frank-tts-engine-warm: voice narrator1 cold=1402ms warm=1812ms frank-tts-engine-warm: 3 voice(s) registered and warm ``` -**2. The thread law → nothing is ever pinned.** `tts-server` in this build has **no `-t`/`--threads` flag**: `src/backend.h` computes the CPU thread count as `std::thread::hardware_concurrency() / 2` (its comment: GEMM shares SIMD units across hyperthreads). Thread count therefore follows what the kernel *reports*, not what a cgroup *allows* — so `CPUQuota=`, `CPUAffinity=`, `taskset` or a slice silently changes the engine's GEMM tiling and its timings, with no log line. Nothing in this fleet pins this container. If a neighbour starves it, fix the neighbour. +**2. The thread law → nothing is ever pinned.** `tts-server` in this build has **no `-t`/`--threads` flag**: `src/backend.h` computes the CPU thread count as `std::thread::hardware_concurrency() / 2` (its comment: GEMM shares SIMD units across hyperthreads). Thread count therefore follows what the kernel *reports*, not what a cgroup *allows* — so `CPUQuota=`, `CPUAffinity=`, `taskset` or a slice silently changes the engine's pool sizes and its timings, with no log line. The rule did not retire with the iGPU cutover: the binary still sizes those pools from what the kernel says, whatever the device ends up doing with the matmuls. Nothing in this fleet pins this container. If a neighbour starves it, fix the neighbour. **3. Streaming is not buffered.** At the same seed, `response_format=pcm` (chunked, codec chunks pushed as they decode) and `response_format=wav` (buffered) differ in **92.2 % of samples**, −41.6 dB RMS. Nothing may compare the two paths, cache one against the other, or assume a seed pins output across `response_format`. `frank-tts` always asks for `wav`; the warmer asks for `wav`; the test seam is forbidden from pinning either. +**4. The device law → the card is shared, and the rollback is one line.** Synthesis computes on the gfx1151 **beside** [[frank-dragoon]], so a change to either tenant is a change to both. Two gates decide that pair, and both passed on 2026-10-19 with the other tenant live: the RADV heap a rootless container tenant is actually handed (not the ~2 GiB DRM carve-out), and dragoon's own first-token and decode rates with a synthesis in flight — unchanged. Their numbers, the recipe that measures them, and the full change surface are §5 of `report/tts-npu-spike-verdict-2026-10-19.md`; do not re-derive them here. What belongs on this page is the move: `QWENTTS_DEVICE=` (empty) in `scripts/frank-tts-engine.env`, re-deploy this one service, and synthesis is back on CPU — no rebuild, and dragoon never restarts. + ## The voice registry contract A voice becomes known to the engine through `POST /v1/audio/voices`, and only these shapes are accepted: @@ -97,9 +93,9 @@ The **façade** on 7772 is the pair's public surface and carries the probe. Note | `dankclaw/voiceservice/Containerfile.engine` | two-stage build of the pinned commit, proven cmake line, digest-pinned base, ships both scripts | | `dankclaw/voiceservice/tts-engine-start.sh` | image entrypoint (`/app/start.sh`): env → flags, refuses to boot without weights | | `dankclaw/voiceservice/tts-engine-warm.sh` | boot warmer (`/app/warm-voices.sh`): register from `reference.wav` (body via a temp file), speak a fixed sentence twice, report cold/warm | -| `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, health probe, warm hook, `Restart=always` | +| `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, render access (`AddDevice=/dev/dri` + `GroupAdd=keep-groups`), health probe, warm hook, `Restart=always` | | `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the private 10.89.5.0/24 bridge, with the subnet registry | -| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law and the registry law in its header | +| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law, the registry law and the device law in its header | | `scripts/download-frank-tts-engine-qwentts.sh` | weight fetcher: no resume, byte count + LFS oid check, `--verify-only` | | `scripts/deploy-remote.sh` → `deploy_frank_tts_engine` | weights gate → build → quadlet → restart → container health → warm | | `tests/frank_tts_engine.bats` | the seam: generator output, entrypoint argv, warmer against a stub engine, planner and routing maps | @@ -118,6 +114,25 @@ still open. 6. **Then the façade on 7772** (its own page), after the engine answered — that order is the whole point of step 6: publishing the façade in front of a dead engine would make every consumer fail at the first voice line. 7. **Open:** [[Open Notebook]] is still not installed, so nothing renders a podcast yet; the voice leg it waits on now exists. +### 2026-10-19 — the iGPU cutover (PR #34, `6248f02`) + +1. **Configure gate before touching anything** — the flag set, configured inside the + digest-pinned base on Dankclaw: `Found Vulkan: … found components: glslc` then + `Including Vulkan backend`. That probe is what caught `glslc` missing from + `glslang-tools`; the Containerfile's flag comment carries the finding. +2. **Deploy** — `./scripts/deploy.sh dankclaw --service frank-tts-engine`: image rebuild + (the `cfhash` moved), unit reinstall, health gate, warm pass + (`3 voice(s) registered and warm`), Playwright e2e 33/33. +3. **Prove the device, not the flag** — `sudo fuser -v /dev/dri/renderD129` lists + `tts-server` next to dragoon's `flash_serve`. Do this instead of reading the log: the + engine prints nothing about which backend it chose, so argv (`--device Vulkan0`) proves + intent and the render-node holder list proves use. +4. **Consumer path** — one `/v1/audio/speech` per voice through the façade on 7772; all + three answered with audio. The RTF figure in the Facts table comes from one engine-direct + WAV, where the duration is exact. +5. **Co-tenancy gate** — §5 of the verdict report: dragoon's first token and decode + unchanged with a 16.9 s synthesis occupying the card. + ## Troubleshooting | Symptom | Meaning | Move | @@ -131,3 +146,5 @@ still open. | `podman exec` in the deploy says `no such file or directory` for `/app/warm-voices.sh` | the image predates the warmer | re-deploy: since 2026-10-18 `build_compose_image` hashes every file the image `COPY`s, not only `Containerfile.engine`, so editing the warmer rebuilds it — see | | The warm pass logs `registered, but the warm synthesis failed` for EVERY voice, while a hand-run `/app/warm-voices.sh` succeeds | `WARM_TEXT` arrived **with its quotes**. podman's `--env-file` does no shell parsing — it takes the literal remainder of the line — so the env-file reader got `"Warm up, one two three."` where `source` gives `Warm up, one two three.`; the quotes plus a backslash then make the synthesis body invalid JSON | the hook strips one matched pair and escapes what is left; prove the value with `podman exec frank-tts-engine env \| grep WARM_TEXT`. The seam pins both readers, because a test that only sources the env file cannot see this | | A `502` for `/v1/voices` where the engine answers `200` on `/health` from the same boot | the engine restarted and its in-memory registry is empty again — `/health` is model-loaded, NOT voices-registered | re-run `podman exec frank-tts-engine /app/warm-voices.sh`; that is the same distinction the `frank-tts` healthcheck enforces | +| Unit restarts every 10 s and the log ends on a **device** error *after* both GGUFs were read | the image has the Vulkan loader but no ICD, or the container has no render node — the device name then resolves to nothing | `podman exec frank-tts-engine ls /dev/dri` (it must see `renderD129`) and `podman exec frank-tts-engine dpkg -l \| grep -E 'mesa-vulkan\|libvulkan1'`; re-deploy to rebuild the image, or set `QWENTTS_DEVICE=` and re-deploy for the CPU shape | +| `frank-dragoon` slows while TTS speaks | the contention premise broke — a Mesa bump, a third GPU tenant, a bigger synthesis | `QWENTTS_DEVICE=` + re-deploy this service, then re-run the gate recipe in §5 of `report/tts-npu-spike-verdict-2026-10-19.md` before putting it back | diff --git a/serverAppConfigs-docs/Dankclaw/Frank TTS.md b/serverAppConfigs-docs/Dankclaw/Frank TTS.md index face310..06dabe9 100644 --- a/serverAppConfigs-docs/Dankclaw/Frank TTS.md +++ b/serverAppConfigs-docs/Dankclaw/Frank TTS.md @@ -113,17 +113,18 @@ shows up as a failed synthesis, never as a down probe. ## Where the engine computes -CPU, and only CPU is compiled in: `Containerfile.engine` builds `qwentts.cpp` with -`-DGGML_VULKAN=OFF` written out explicitly, so the container carries `libggml-cpu` and no -other backend. The realistic accelerator is the gfx1151 iGPU behind that flag — plus -`/dev/dri` passthrough, the ComfyUI pattern — on the same payload and the same pinned -commit, and the intended shape is **co-tenancy**: the engine resident on the card beside -`frank-dragoon`, not a handover. Memory is not what decides that; shader contention is, and -the census plus both gates are in `report/tts-npu-spike-verdict-2026-10-19.md` §5. -**The Strix Halo NPU is not a door at all**: XDNA2 executes only precompiled AIE -partition programs, and AMD's own Linux runtime opens `/dev/accel/accel0`, succeeds at the -ioctls, then declines this model family's graph without emitting a single log line. The -measured record and the model-side contract live in `report/` at the repo root +The **gfx1151 iGPU**, as a co-tenant beside `frank-dragoon`: the Vulkan build of +`qwentts.cpp` with `--device Vulkan0`, and `/dev/dri` handed to the engine container. That is +the shape since 2026-10-19. The flag, the device name, the render-node trap on this hybrid box +and the one-line CPU rollback are owned by [[Frank TTS Engine]] — read that page before +touching either tenant, because a change to one is a change to both. Co-tenancy was gated with +`frank-dragoon` live on the card (its first token and decode rates unchanged with a synthesis +in flight); both gate recipes are §5 of `report/tts-npu-spike-verdict-2026-10-19.md`. + +**The Strix Halo NPU is not a door at all**: XDNA2 executes only precompiled AIE partition +programs, and AMD's own Linux runtime opens `/dev/accel/accel0`, succeeds at the ioctls, then +declines this model family's graph without emitting a single log line. The measured record and +the model-side contract live in `report/` at the repo root (`tts-npu-spike-verdict-2026-10-19.md`). ## Files diff --git a/serverAppConfigs-docs/index.md b/serverAppConfigs-docs/index.md index 145eb6b..cbba6a7 100644 --- a/serverAppConfigs-docs/index.md +++ b/serverAppConfigs-docs/index.md @@ -23,7 +23,7 @@ is a locator: one row, one clause, no numbers. | [[ComfyUI Qwen-Image Workflows]] | Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next. | | [[Dankclaw Stack]] | Service inventory and routing overview for the Dankclaw AI, development, and observability host. | | [[Frank TTS]] | The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role. | -| [[Frank TTS Engine]] | The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes. | +| [[Frank TTS Engine]] | The engine behind the fleet's speech endpoint: its Vulkan build on the iGPU, its private bridge, and what it publishes. | | [[Gitea]] | Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service. | | [[Gluetun]] | Gluetun is a lightweight VPN client container. | | [[Hermes]] | Hermes is the self-improving AI agent from Nous Research. |