dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(frank-tts-engine): synthesize on the gfx1151 iGPU via Vulkan

Browse Source 
The NPU path is a dead end for this payload (verdict:
report/tts-npu-spike-verdict-2026-10-19.md); the iGPU is the accelerator that
exists on this box today, and it is already shared with frank-dragoon.

- Containerfile.engine: -DGGML_VULKAN=ON, plus the three packages the door
  needs to configure — libvulkan-dev, spirv-headers, and glslc, which Ubuntu's
  glslang-tools does NOT ship (find_package(Vulkan COMPONENTS glslc REQUIRED)
  fails without it) — and libvulkan1 + mesa-vulkan-drivers (the RADV ICD) at
  runtime. No NVIDIA ICD on purpose: one Vulkan device in the enumeration is
  what makes the NAME Vulkan0 safe to write down.
- tts-engine-start.sh: QWENTTS_DEVICE -> --device, default Vulkan0. A value
  that is set but EMPTY passes no flag at all, which is the CPU rollback; that
  is why the default is written as an -v test and not with :=, because := also
  fires on an empty value and would silently undo the rollback.
- quadlet: AddDevice=/dev/dri + GroupAdd=keep-groups (a rootless container that
  drops its supplementary groups enumerates no render device). The node is not
  named: this host is hybrid, renderD128 is the NVIDIA card and renderD129 the
  amdgpu iGPU.
- env: QWENTTS_DEVICE=Vulkan0 plus the device law in the header, including the
  rollback line.
- tests: the two CPU-only assertions now assert the Vulkan shape, and a new one
  pins the rollback (empty device => no --device in argv). 451/451 green.

Measured before writing any of it: the configure in the digest-pinned base
answers "Found Vulkan: ... found components: glslc" and "Including Vulkan
backend"; gate 1 (RADV GFX1151 visible to a rootless container off this base)
is recorded in the verdict report.
This commit is contained in:
Frank
2026-10-19 14:26:58 -07:00
parent 7601edfe28
commit 25df9d05a2
5 changed files with 179 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dankclaw/voiceservice/Containerfile.engine
+44 -6
View File
@@ -19,11 +19,14 @@
# is recorded in the parent tree, so cloning the parent commit recursively is
# reproducible without a second pin.
#
# THE BUILD LINE IS THE ONE PROVEN ON DANKCLAW, verbatim:
# THE BUILD LINE IS THE ONE PROVEN ON DANKCLAW, verbatim — with the Vulkan door
# open since 2026-10-19. That configure was run in this exact base image and
# answers `Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so (found version
# "1.3.275") found components: glslc` then `Including Vulkan backend`:
#
# cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CCACHE=OFF \
# -DGGML_BACKEND_DL=OFF -DGGML_CPU_ALL_VARIANTS=OFF -DGGML_OPENMP=ON \
# -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=OFF \
# -DGGML_BLAS=OFF -DGGML_CUDA=OFF -DGGML_HIP=OFF -DGGML_VULKAN=ON \
# -DGGML_METAL=OFF -DGGML_SYCL=OFF -DGGML_RPC=OFF \
# && cmake --build build -j 8
#
@@ -42,8 +45,12 @@
# defaults it ON outside wasm/cross-compiles, i.e. -march=native for THIS
# host — correct while build host == run host, wrong if anyone ever builds
# this image elsewhere, which is why the fleet builds it on Dankclaw.
# CUDA/HIP/VULKAN/METAL/SYCL/RPC all OFF: this is a CPU endpoint (the trial
# ran on CPU; the box's GPUs belong to the text endpoints).
# CUDA/HIP/METAL/SYCL/RPC stay OFF, VULKAN is ON: the endpoint computes on the
# gfx1151 iGPU (decision and co-tenancy gates:
# report/tts-npu-spike-verdict-2026-10-19.md, shape in [[Frank TTS Engine]]).
# Nothing else is available to this image: no ROCm userland, no NVIDIA
# userland, no macOS, no oneAPI, and RPC would add a second process to
# supervise on a box that already has one engine per card.
#
# BUILD SHARED LIBS, RUNTIME SMALL. BUILD_SHARED_LIBS defaults ON on Linux, so
# tts-server links libggml*.so rather than swallowing them; the runtime stage
@@ -67,9 +74,28 @@ ARG BASE_IMAGE=docker.io/library/ubuntu:24.04@sha256:68ee22004a9109b44973313b02f
# ------------------------------------------------------------------ build
FROM ${BASE_IMAGE} AS build
# The three Vulkan build deps are not optional and not interchangeable:
# libvulkan-dev vulkan/vulkan.h + the VulkanHeaders/VulkanLoader CMake
# configs, i.e. what find_package(Vulkan) resolves to.
# glslc ggml compiles its shaders with glslc, not glslangValidator.
# Ubuntu's glslang-tools ships glslangValidator and NOT glslc,
# so without this package `find_package(Vulkan COMPONENTS
# glslc REQUIRED)` fails and there is no Vulkan backend.
# The archive's glslc is old (2023.8): it compiles
# GL_KHR_cooperative_matrix — the path ggml's quantized
# matmul needs — but reports cooperative_matrix2,
# integer_dot_product, bfloat16 and the micro-float
# extensions as unsupported, so those shader families are
# compiled out. If a Vulkan prefill ever reads as
# inexplicably slow, suspect this compiler before the device;
# the fix is a pinned glslang build passed as
# -DVulkan_GLSLC_EXECUTABLE=<path>, not a different runtime.
# spirv-headers find_package(SPIRV-Headers CONFIG REQUIRED); the distro
# package ships /usr/share/cmake/SPIRV-Headers/.
RUN apt-get update -qq \
&& apt-get install -y -qq --no-install-recommends \
ca-certificates cmake g++ git make \
libvulkan-dev glslc spirv-headers \
&& rm -rf /var/lib/apt/lists/*
# Checked out detached at a full SHA, then the recorded submodule. No branch
@@ -90,7 +116,7 @@ RUN cmake -B build -DCMAKE_BUILD_TYPE=Release \
-DGGML_BLAS=OFF \
-DGGML_CUDA=OFF \
-DGGML_HIP=OFF \
-DGGML_VULKAN=OFF \
-DGGML_VULKAN=ON \
-DGGML_METAL=OFF \
-DGGML_SYCL=OFF \
-DGGML_RPC=OFF \
@@ -102,9 +128,21 @@ FROM ${BASE_IMAGE}
# libgomp1: the OpenMP runtime the build above links against. curl: the unit's
# HealthCmd. ca-certificates: so that curl can speak https if anyone ever
# points a probe at a URL with a scheme this host terminates elsewhere.
#
# libvulkan1 + mesa-vulkan-drivers are what make `--device Vulkan0` mean
# anything at runtime: libvulkan1 is the loader libggml-vulkan.so dlopens, and
# mesa-vulkan-drivers is the RADV ICD that claims the gfx1151. Loader without an
# ICD enumerates ZERO devices, and tts-server then dies in device lookup AFTER
# both GGUFs have been read — the journal reads as a model fault, not a missing
# driver. Enumerated from this base on 2026-10-19 with `--device /dev/dri
# --group-add keep-groups` plus the host's ComfyUI udev rule: "Radeon 8060S
# Graphics (RADV GFX1151)", API 1.4.318 (the heap figure is owned by
# [[Frank TTS Engine]]; it is the GTT-backed aperture, not the ~2 GiB DRM
# carve-out). No NVIDIA ICD is installed on purpose: one Vulkan device in the
# enumeration is what makes the NAME Vulkan0 safe to write down.
RUN apt-get update -qq \
&& apt-get install -y -qq --no-install-recommends \
ca-certificates curl libgomp1 \
ca-certificates curl libgomp1 libvulkan1 mesa-vulkan-drivers \
&& rm -rf /var/lib/apt/lists/*
# Same unprivileged uid as the frank-tts façade (10000): the voice packs and the
dankclaw/voiceservice/quadlet/frank-tts-engine.container
+20 -6
View File
@@ -40,12 +40,23 @@
# speaks to the server from inside its own namespace, which is where the address
# is valid.
#
# MEMORY / NO GPU. Peak RSS measured 5.7 GiB with both GGUFs resident
# (2026-10-18 trial); no memory limit is set because the box has the room and a
# limit would convert a busy moment into an OOM kill mid-utterance. No
# /dev/kfd, no /dev/dri, no keep-groups: every GPU backend is compiled out of
# the image, the endpoint is CPU-only by decision, and the GPUs on this host
# belong to the text endpoints.
# MEMORY / THE RENDER NODE. Host RSS stays near the 5.7 GiB the CPU trial
# measured (weights + context resident); no memory limit is set because the box
# has the room and a limit would convert a busy moment into an OOM kill
# mid-utterance.
#
# `AddDevice=/dev/dri` + `GroupAdd=keep-groups` are what make the iGPU visible,
# and both are required: a rootless container that drops its supplementary
# groups has no render permission, and the Vulkan ICD then enumerates nothing —
# this is the same ride ComfyUI takes (dankclaw/comfyui, plus the udev rule that
# makes the nodes group-writable). The node is NOT named: this host is hybrid,
# renderD128 is the NVIDIA card and renderD129 the amdgpu iGPU, so a node number
# in the unit is a guess that a re-probe can invalidate, while the directory
# hands the container every render node and the driver picks by name.
#
# /dev/kfd stays absent on purpose: that is the ROCm compute interface, owned by
# the engines that use HIP. A Vulkan endpoint has no use for it, and handing a
# container a second way onto the GPU is surface for nothing.
#
# THE VOICE MOUNT IS READ-ONLY AND IS FOR THE WARMER, NOT THE ENGINE. A voice
# becomes known to the engine only through POST /v1/audio/voices, and the
@@ -98,6 +109,9 @@ Network=frank-tts-engine.network
EnvironmentFile=/home/dank/serverAppConfigs/scripts/frank-tts-engine.env
Volume=/home/dank/models/Qwen3-TTS-GGUF:/models:ro
Volume=/home/dank/voices:/voices:ro
# Render access for the iGPU (see MEMORY / THE RENDER NODE above).
AddDevice=/dev/dri
GroupAdd=keep-groups
HealthCmd=curl -fsS http://127.0.0.1:8880/health
HealthInterval=30s
HealthTimeout=10s
dankclaw/voiceservice/tts-engine-start.sh
+54 -19
View File
@@ -7,7 +7,7 @@
# supervision, no retries, no logging of its own. PID 1 stays the server, so
# `podman stop` and the unit's Restart=always drive it directly.
#
# THE FLAGS THAT ARE NOT HERE, and must stay absent:
# THE FLAG THAT MUST STAY ABSENT:
#
# --threads / -t does not exist in this build. src/backend.h computes the
# CPU thread count as std::thread::hardware_concurrency()/2
@@ -18,13 +18,19 @@
# placement — pinning is deliberately not done anywhere in
# this stack. See the env file header for the same warning
# aimed at the operator.
# --device left at its default. Every GPU backend is compiled out of
# the image, so the default device pick can only land on
# CPU; naming a device would add a way to be wrong.
#
# Everything defaults to the shape the 2026-10-18 trial measured, so an empty
# environment still boots the decided configuration: 1.7 B talker Q4_K_M with
# the Q8_0 tokenizer, one request at a time, on the private bridge.
# --device IS here, but conditionally: it reaches the binary only when
# QWENTTS_DEVICE is non-empty, and empty is the CPU rollback — the binary then
# takes its own default device, which is CPU because the CPU backend is always
# built. One line in scripts/frank-tts-engine.env moves the endpoint between the
# two shapes, with no rebuild. What must never reach argv is a device NAME the
# loader does not enumerate: that is not refused by the argument parser, it dies
# during device lookup with the loader's own error, so the journal reads as a
# load that failed rather than a configuration refusal.
#
# Everything defaults to the shape this endpoint runs: 1.7 B talker Q4_K_M with
# the Q8_0 tokenizer, one request at a time, on the private bridge, on the
# gfx1151 iGPU. An empty environment still boots that shape.
set -euo pipefail
# The binary path. Overridable for one reason: the bats seam runs a stub
@@ -60,13 +66,33 @@ set -euo pipefail
: "${QWENTTS_LANG:=auto}"
# One request at a time. The server-side queue is a serialised FIFO at this
# setting, which is what keeps the measured per-voice timings true; batching
# on a CPU build splits the same cores between two syntheses and the warm RTF
# margin (0.55-0.67 against real time 1.0) disappears. The engine's own
# default is also 1; it is written here because it is a decision, not an
# accident.
# setting, which is what keeps the per-voice timings predictable. A second
# concurrent synthesis shares the device with the first — and on this box that
# device is also carrying frank-dragoon — so it buys no throughput and it
# doubles the wait for the caller behind it. The engine's own default is also 1;
# written here because it is a decision, not an accident.
: "${QWENTTS_MAX_BATCH:=1}"
# The compute device, NAMED and not numbered: `Vulkan0` is how ggml enumerates
# the gfx1151 iGPU on this host. An index would follow whatever order the
# loader's device list returns; a name says which device this endpoint means.
# The container sees exactly one Vulkan device because the image carries the
# Mesa ICD and no NVIDIA one, so the 3090 is not in the enumeration at all.
#
# EMPTY IS THE CPU ROLLBACK: a value that is set but EMPTY passes no `--device`,
# the binary picks its own default, and the CPU backend is always built — which
# is why moving this endpoint between the iGPU and the CPU is one line in
# scripts/frank-tts-engine.env and not a rebuild.
#
# That is also why the default below is NOT `${QWENTTS_DEVICE:=Vulkan0}`: `:=`
# fires on a set-but-empty variable as well as an unset one, so the rollback line
# `QWENTTS_DEVICE=` would be rewritten back to Vulkan0 and the endpoint would sit
# on the shared GPU with no way out short of a rebuild. Only UNSET takes the
# default.
if [[ ! -v QWENTTS_DEVICE ]]; then
QWENTTS_DEVICE=Vulkan0
fi
# Fail loudly BEFORE binding, so the journal says which artifact is missing
# instead of the unit reporting a container that starts and dies. This is the
# self-healing path: Restart=always re-runs it every RestartSec, and the same
@@ -79,11 +105,20 @@ for artifact in "$QWENTTS_MODEL" "$QWENTTS_CODEC"; do
fi
done
exec "$QWENTTS_BIN" \
--model "$QWENTTS_MODEL" \
--codec "$QWENTTS_CODEC" \
--alias "$QWENTTS_ALIAS" \
--host "$QWENTTS_HOST" \
--port "$QWENTTS_PORT" \
--lang "$QWENTTS_LANG" \
ARGS=(
--model "$QWENTTS_MODEL"
--codec "$QWENTTS_CODEC"
--alias "$QWENTTS_ALIAS"
--host "$QWENTTS_HOST"
--port "$QWENTTS_PORT"
--lang "$QWENTTS_LANG"
--max-batch "$QWENTTS_MAX_BATCH"
)
# Only when a device is named. An empty QWENTTS_DEVICE has to produce NO flag,
# because "the rollback shape" is argv identical to the pre-iGPU endpoint.
if [[ -n "$QWENTTS_DEVICE" ]]; then
ARGS+=(--device "$QWENTTS_DEVICE")
fi
exec "$QWENTTS_BIN" "${ARGS[@]}"