dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(frank-tts): record the iGPU cutover, close the NPU question

Browse Source 
The cutover PR merged the runtime; this carries the surface that has to be
true afterwards.

- Engine page: the shape is now `--device Vulkan0` on the gfx1151 as a
  co-tenant beside frank-dragoon; the registry/thread/streaming laws gain a
  fourth — the device law — whose numbers stay in the verdict report, and the
  deploy record gets the cutover steps including the `glslc` configure gate.
  The CPU-era arm/rollback prose keeps the qualitative argument and drops the
  dead numbers per the purpose test.
- Endpoint page: "Where the engine computes" now says iGPU + co-tenancy, and
  states the NPU verdict in one paragraph instead of a research tour.
- Quadlet comment: it claimed supplementary groups are what grant render
  access. On this host the udev rule world-writes the render nodes and the
  container's `id` shows no groups at all, so keep-groups is belt-and-braces;
  the RSS note now matches the device shape instead of the CPU trial.
- Verdict report: gate 2 (contention) recorded as measured after the cutover,
  and the build-stage prediction corrected — the package is `glslc`, not
  `libvulkan-dev glslang-dev`.
- index.md regenerated (one description changed).
This commit is contained in:
Frank
2026-10-19 14:37:29 -07:00
parent 6248f02ca2
commit ea2fc6440b
5 changed files with 98 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md
+40 -23
View File
@@ -1,35 +1,36 @@
---
type: reference
description: "The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes."
description: "The engine behind the fleet's speech endpoint: its Vulkan build on the iGPU, its private bridge, and what it publishes."
tags:
- dankclaw
- tts
- voice
- quadlet
- cpu
timestamp: 2026-10-18
- vulkan
timestamp: 2026-10-19
---
# Frank TTS Engine
`frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** in RAM once and synthesizing from there. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page.
`frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** loaded once and synthesizing from there — **on the gfx1151 iGPU** (ggml's Vulkan backend, `--device Vulkan0`) since 2026-10-19, and in host RAM before that. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 when asked, and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page.
**Deployed 2026-10-18.** The chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified, 18 s), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy 31 s after the build, **4.0 GiB** resident on a host that kept 17 GiB available, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user.
**Deployed 2026-10-18, cut over to the iGPU on 2026-10-19.** The first chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy shortly after the build, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user.
**Host:** Dankclaw (`192.168.0.143`), rootless Podman, user `dank` — **Port:** 8880, **never published** — **Network:** the private `systemd-frank-tts-engine` bridge (10.89.5.0/24) and no other — **Deployer:** `frank-tts-engine` — **Consumers:** `frank-tts` (7772) and, through it, every voice request in the fleet including [[Open Notebook]]'s podcast renders.
## Facts (verified 2026-10-18 against the pinned upstream commit, the Hub, and the CPU trial on this host)
## Facts (verified 2026-10-18 against the pinned upstream commit, the Hub and the CPU trial; the device rows re-verified on the live iGPU shape 2026-10-19)
| Item | Value |
|------|-------|
| Engine | `tts-server` from qwentts.cpp, pinned at commit `5916bf7c79dbb4e66fc059cdc507a79a23a28bb4` (`dankclaw/voiceservice/Containerfile.engine`). The source is **not vendored**: the image clones that commit and its recorded `ggml` submodule, so the build needs outbound HTTPS and the `cfhash` label is what keeps an unchanged re-deploy offline |
| Build line | `cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF && cmake --build build -j 8` — the exact line the 2026-10-18 trial ran on this box. `GGML_BLAS=OFF` is **forced**: Dankclaw has `libopenblas.so.0` and no BLAS dev headers, so upstream's own `Dockerfile` and `buildcpu.sh` (both of which pass BLAS on) cannot be copied. No `-DGGML_NATIVE` is passed, and ggml defaults it on, so the binary is `-march=native` for the machine that built it: **build this image on the host that runs it** |
| Image | built locally as `franksdank/frank-tts-engine:latest`, two stages from one digest-pinned base `docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac` (index digest resolved 2026-10-18). Runtime stage: `libgomp1` (OpenMP is on), `curl` (the unit's `HealthCmd` runs as `/bin/sh -c`, and the warmer speaks HTTP), uid **10000**, `LD_LIBRARY_PATH=/app` because ggml builds shared by default and the link-time RPATH names the build tree |
| Build line | `cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=ON -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF && cmake --build build -j 8`. **`GGML_VULKAN=ON` since 2026-10-19**, and that door needs three packages in the build stage: `libvulkan-dev`, `spirv-headers`, and **`glslc`** — which Ubuntu's `glslang-tools` does NOT ship (it ships `glslangValidator`, and `find_package(Vulkan COMPONENTS glslc REQUIRED)` fails without the package). The archive's `glslc` is 2023.8: `GL_KHR_cooperative_matrix` compiles, the coopmat2 / integer-dot / bf16 / micro-float shader families do not. `GGML_BLAS=OFF` is **forced**: Dankclaw has `libopenblas.so.0` and no BLAS dev headers, so upstream's own `Dockerfile` and `buildcpu.sh` (both pass BLAS on) cannot be copied. No `-DGGML_NATIVE` is passed, and ggml defaults it on, so the binary is `-march=native` for the machine that built it: **build this image on the host that runs it** |
| Image | built locally as `franksdank/frank-tts-engine:latest`, two stages from one digest-pinned base `docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02ff7b1de250f1de0f5ecb628d73f579aaff33ac` (index digest resolved 2026-10-18). Runtime stage: `libgomp1` (OpenMP is on), `curl` (the unit's `HealthCmd` runs as `/bin/sh -c`, and the warmer speaks HTTP), `libvulkan1` + `mesa-vulkan-drivers` (the loader and the RADV ICD — a loader with no ICD enumerates ZERO devices, and the server then dies in device lookup after both GGUFs have been read), uid **10000**, `LD_LIBRARY_PATH=/app` because ggml builds shared by default and the link-time RPATH names the build tree. **No NVIDIA ICD on purpose**: exactly one Vulkan device in the enumeration is what makes the name `Vulkan0` safe to write down |
| Talker | `qwen-talker-1.7b-base-Q4_K_M.gguf`, **1 219 245 248 B**, sha256 `ea393ebaf2167ea23ce9fc18b093822851358a950d7075cd47ab4f6ce23e887d` (the Hub's LFS pointer oid) |
| Tokenizer | `qwen-tokenizer-12hz-Q8_0.gguf`, **291 150 624 B**, sha256 `1883beeed99348fc35e23dd225e9082f93f6f8c109330a33d935baa8acdbfd94` |
| Source of both | `https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF`, fetched by `scripts/download-frank-tts-engine-qwentts.sh` into `/home/dank/models/Qwen3-TTS-GGUF`, mounted read-only at `/models`. It has to be the **base** variant: the engine refuses every voice registration unless the model reports type `base`, so on `customvoice`/`voicedesign` the cloned-voice product does not exist. A 0.6 B talker (628 905 056 B) exists and is **not** the decided artifact |
| Memory | **5.7 GiB peak RSS** with both files resident (2026-10-18 trial). No `LimitMEMORY`/`MemoryMax`: a limit would turn a busy moment into an OOM kill in the middle of an utterance |
| Concurrency | `--max-batch 1`: a serialised FIFO. Warm RTF on this box is 0.55–0.67 against real time 1.0, and that margin is the entire headroom a live stream has; two concurrent syntheses split the same cores and the second caller's first byte additionally waits for the first utterance to finish |
| Device | `--device Vulkan0`: the gfx1151 iGPU, **shared with [[frank-dragoon]] as a co-tenant, never a handover**. Proven on the live pair, not inferred: `fuser -v /dev/dri/renderD129` lists `tts-server` beside `flash_serve`. Both gates (the RADV heap and dragoon's contention) are §5 of `report/tts-npu-spike-verdict-2026-10-19.md`, which also carries the change surface. **Rollback: `QWENTTS_DEVICE=` (empty) in `scripts/frank-tts-engine.env`, redeploy this one service** — the entrypoint then passes no `--device` at all, which is the CPU shape, no rebuild, nothing else restarts |
| Memory | **1.7 GiB host RSS** since the iGPU cutover — the weights ride the device. No `LimitMEMORY`/`MemoryMax`: a limit would turn a busy moment into an OOM kill in the middle of an utterance |
| Concurrency | `--max-batch 1`: a serialised FIFO. Warm RTF on the iGPU measures **0.20** (11.04 s of audio in 2.23 s) with `frank-dragoon` live on the same card, so a live stream has room to spare; a second concurrent synthesis would share the device with the first and make the caller behind it wait out the whole first utterance, so the FIFO stays a decision, not an accident |
| Network | `systemd-frank-tts-engine` (10.89.5.0/24, `dankclaw/voiceservice/quadlet/frank-tts-engine.network`) and **nothing else**, with **no `PublishPort` at all**. `frank-tts` joins the same bridge and dials `http://frank-tts-engine:8880` |
| Voices | `/home/dank/voices` mounted read-only at `/voices`, and it is the **warmer** that reads it, not the server: `tts-server` receives reference audio as base64 in a registration body and keeps its registry in memory only |
| Health | `GET /health` → `{"status":"ok"}`, served by the HTTP thread rather than the synthesis queue, so a busy engine still reads healthy. Probed by the unit (`HealthCmd`, 30 s interval, 120 s start period) **inside the container's namespace**, because there is no published port to probe from outside |
@@ -37,18 +38,11 @@ timestamp: 2026-10-18
## Why this engine, and not the alternatives
The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. RTF only has to stay under 1.0 with margin; what separates the arms is first-byte latency and **spread**. Measured on this host the same day (CPU; the full A/B is the section *"Decision: arm A for live output (2026-10-18)"* in , which is the source of these numbers):
The criterion Frank set on 2026-10-18 was **"whatever is better for live output"**, and live output is decided by time-to-first-audio and by whether the stream stays ahead of real time — not by benchmark throughput. On that criterion the resident `tts-server` (arm A) beat the mainline per-request CLI path (arm B) on this box the same day, and the deciding property was **spread**: the resident server repeats the same first-audio latency every run, while the CLI path varies enough that some runs fall behind real time and a live listener hears the stream stall mid-sentence. That A/B was a CPU measurement and the endpoint has since moved to the iGPU, so the shape of the argument is what is kept here; no page carries its numbers any more, and they are not worth re-deriving.
| Measure | Resident `tts-server` (arm A, chosen) | Mainline per-request CLI path (arm B) |
|---|---|---|
| first audio byte | **86–451 ms** end to end; **94–112 ms** server-side | model load, per request |
| warm real-time factor while streaming | **0.55–0.67** | 0.93–0.99 |
| run-to-run spread | **±0.01** | ±0.25 |
| HTTP surface | serves `/v1/audio/speech` itself | none — façade CLI only |
The rollback to the other backend stays in the repo untouched — `TTS_BACKEND=llama-tts-cli` in `scripts/frank-tts.env` still selects the per-request CLI engine, and that file's header documents what it costs.
The spread is the whole argument: at ±0.01 the engine finishes synthesizing 1.5–1.8× faster than the audio plays, every run; at ±0.25 some runs fall behind real time and a live listener hears the stream stall mid-sentence. The rollback path stays in the repo untouched — `TTS_BACKEND=llama-tts-cli` in `scripts/frank-tts.env` still selects the per-request CLI engine, and that file's header documents what it costs.
## Three laws this service lives under
## Four laws this service lives under
**1. The registry law → the unit warms at boot.** The voice registry is **memory only**: a restart forgets every voice, and re-registering one does **not** warm it — only synthesizing does. A boot that registers nothing leaves the façade answering 400 for every name, which is why `frank-tts-engine.container` carries
@@ -64,10 +58,12 @@ frank-tts-engine-warm: voice narrator1 cold=1402ms warm=1812ms
frank-tts-engine-warm: 3 voice(s) registered and warm
```
**2. The thread law → nothing is ever pinned.** `tts-server` in this build has **no `-t`/`--threads` flag**: `src/backend.h` computes the CPU thread count as `std::thread::hardware_concurrency() / 2` (its comment: GEMM shares SIMD units across hyperthreads). Thread count therefore follows what the kernel *reports*, not what a cgroup *allows* — so `CPUQuota=`, `CPUAffinity=`, `taskset` or a slice silently changes the engine's GEMM tiling and its timings, with no log line. Nothing in this fleet pins this container. If a neighbour starves it, fix the neighbour.
**2. The thread law → nothing is ever pinned.** `tts-server` in this build has **no `-t`/`--threads` flag**: `src/backend.h` computes the CPU thread count as `std::thread::hardware_concurrency() / 2` (its comment: GEMM shares SIMD units across hyperthreads). Thread count therefore follows what the kernel *reports*, not what a cgroup *allows* — so `CPUQuota=`, `CPUAffinity=`, `taskset` or a slice silently changes the engine's pool sizes and its timings, with no log line. The rule did not retire with the iGPU cutover: the binary still sizes those pools from what the kernel says, whatever the device ends up doing with the matmuls. Nothing in this fleet pins this container. If a neighbour starves it, fix the neighbour.
**3. Streaming is not buffered.** At the same seed, `response_format=pcm` (chunked, codec chunks pushed as they decode) and `response_format=wav` (buffered) differ in **92.2 % of samples**, −41.6 dB RMS. Nothing may compare the two paths, cache one against the other, or assume a seed pins output across `response_format`. `frank-tts` always asks for `wav`; the warmer asks for `wav`; the test seam is forbidden from pinning either.
**4. The device law → the card is shared, and the rollback is one line.** Synthesis computes on the gfx1151 **beside** [[frank-dragoon]], so a change to either tenant is a change to both. Two gates decide that pair, and both passed on 2026-10-19 with the other tenant live: the RADV heap a rootless container tenant is actually handed (not the ~2 GiB DRM carve-out), and dragoon's own first-token and decode rates with a synthesis in flight — unchanged. Their numbers, the recipe that measures them, and the full change surface are §5 of `report/tts-npu-spike-verdict-2026-10-19.md`; do not re-derive them here. What belongs on this page is the move: `QWENTTS_DEVICE=` (empty) in `scripts/frank-tts-engine.env`, re-deploy this one service, and synthesis is back on CPU — no rebuild, and dragoon never restarts.
## The voice registry contract
A voice becomes known to the engine through `POST /v1/audio/voices`, and only these shapes are accepted:
@@ -97,9 +93,9 @@ The **façade** on 7772 is the pair's public surface and carries the probe. Note
| `dankclaw/voiceservice/Containerfile.engine` | two-stage build of the pinned commit, proven cmake line, digest-pinned base, ships both scripts |
| `dankclaw/voiceservice/tts-engine-start.sh` | image entrypoint (`/app/start.sh`): env → flags, refuses to boot without weights |
| `dankclaw/voiceservice/tts-engine-warm.sh` | boot warmer (`/app/warm-voices.sh`): register from `reference.wav` (body via a temp file), speak a fixed sentence twice, report cold/warm |
| `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, health probe, warm hook, `Restart=always` |
| `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, render access (`AddDevice=/dev/dri` + `GroupAdd=keep-groups`), health probe, warm hook, `Restart=always` |
| `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the private 10.89.5.0/24 bridge, with the subnet registry |
| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law and the registry law in its header |
| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law, the registry law and the device law in its header |
| `scripts/download-frank-tts-engine-qwentts.sh` | weight fetcher: no resume, byte count + LFS oid check, `--verify-only` |
| `scripts/deploy-remote.sh` → `deploy_frank_tts_engine` | weights gate → build → quadlet → restart → container health → warm |
| `tests/frank_tts_engine.bats` | the seam: generator output, entrypoint argv, warmer against a stub engine, planner and routing maps |
@@ -118,6 +114,25 @@ still open.
6. **Then the façade on 7772** (its own page), after the engine answered — that order is the whole point of step 6: publishing the façade in front of a dead engine would make every consumer fail at the first voice line.
7. **Open:** [[Open Notebook]] is still not installed, so nothing renders a podcast yet; the voice leg it waits on now exists.
### 2026-10-19 — the iGPU cutover (PR #34, `6248f02`)
1. **Configure gate before touching anything** — the flag set, configured inside the
digest-pinned base on Dankclaw: `Found Vulkan: … found components: glslc` then
`Including Vulkan backend`. That probe is what caught `glslc` missing from
`glslang-tools`; the Containerfile's flag comment carries the finding.
2. **Deploy** — `./scripts/deploy.sh dankclaw --service frank-tts-engine`: image rebuild
(the `cfhash` moved), unit reinstall, health gate, warm pass
(`3 voice(s) registered and warm`), Playwright e2e 33/33.
3. **Prove the device, not the flag** — `sudo fuser -v /dev/dri/renderD129` lists
`tts-server` next to dragoon's `flash_serve`. Do this instead of reading the log: the
engine prints nothing about which backend it chose, so argv (`--device Vulkan0`) proves
intent and the render-node holder list proves use.
4. **Consumer path** — one `/v1/audio/speech` per voice through the façade on 7772; all
three answered with audio. The RTF figure in the Facts table comes from one engine-direct
WAV, where the duration is exact.
5. **Co-tenancy gate** — §5 of the verdict report: dragoon's first token and decode
unchanged with a 16.9 s synthesis occupying the card.
## Troubleshooting
| Symptom | Meaning | Move |
@@ -131,3 +146,5 @@ still open.
| `podman exec` in the deploy says `no such file or directory` for `/app/warm-voices.sh` | the image predates the warmer | re-deploy: since 2026-10-18 `build_compose_image` hashes every file the image `COPY`s, not only `Containerfile.engine`, so editing the warmer rebuilds it — see |
| The warm pass logs `registered, but the warm synthesis failed` for EVERY voice, while a hand-run `/app/warm-voices.sh` succeeds | `WARM_TEXT` arrived **with its quotes**. podman's `--env-file` does no shell parsing — it takes the literal remainder of the line — so the env-file reader got `"Warm up, one two three."` where `source` gives `Warm up, one two three.`; the quotes plus a backslash then make the synthesis body invalid JSON | the hook strips one matched pair and escapes what is left; prove the value with `podman exec frank-tts-engine env \| grep WARM_TEXT`. The seam pins both readers, because a test that only sources the env file cannot see this |
| A `502` for `/v1/voices` where the engine answers `200` on `/health` from the same boot | the engine restarted and its in-memory registry is empty again — `/health` is model-loaded, NOT voices-registered | re-run `podman exec frank-tts-engine /app/warm-voices.sh`; that is the same distinction the `frank-tts` healthcheck enforces |
| Unit restarts every 10 s and the log ends on a **device** error *after* both GGUFs were read | the image has the Vulkan loader but no ICD, or the container has no render node — the device name then resolves to nothing | `podman exec frank-tts-engine ls /dev/dri` (it must see `renderD129`) and `podman exec frank-tts-engine dpkg -l \| grep -E 'mesa-vulkan\|libvulkan1'`; re-deploy to rebuild the image, or set `QWENTTS_DEVICE=` and re-deploy for the CPU shape |
| `frank-dragoon` slows while TTS speaks | the contention premise broke — a Mesa bump, a third GPU tenant, a bigger synthesis | `QWENTTS_DEVICE=` + re-deploy this service, then re-run the gate recipe in §5 of `report/tts-npu-spike-verdict-2026-10-19.md` before putting it back |