feat(frank-tts-engine): synthesize on the gfx1151 iGPU via Vulkan
The NPU path is a dead end for this payload (verdict: report/tts-npu-spike-verdict-2026-10-19.md); the iGPU is the accelerator that exists on this box today, and it is already shared with frank-dragoon. - Containerfile.engine: -DGGML_VULKAN=ON, plus the three packages the door needs to configure — libvulkan-dev, spirv-headers, and glslc, which Ubuntu's glslang-tools does NOT ship (find_package(Vulkan COMPONENTS glslc REQUIRED) fails without it) — and libvulkan1 + mesa-vulkan-drivers (the RADV ICD) at runtime. No NVIDIA ICD on purpose: one Vulkan device in the enumeration is what makes the NAME Vulkan0 safe to write down. - tts-engine-start.sh: QWENTTS_DEVICE -> --device, default Vulkan0. A value that is set but EMPTY passes no flag at all, which is the CPU rollback; that is why the default is written as an -v test and not with :=, because := also fires on an empty value and would silently undo the rollback. - quadlet: AddDevice=/dev/dri + GroupAdd=keep-groups (a rootless container that drops its supplementary groups enumerates no render device). The node is not named: this host is hybrid, renderD128 is the NVIDIA card and renderD129 the amdgpu iGPU. - env: QWENTTS_DEVICE=Vulkan0 plus the device law in the header, including the rollback line. - tests: the two CPU-only assertions now assert the Vulkan shape, and a new one pins the rollback (empty device => no --device in argv). 451/451 green. Measured before writing any of it: the configure in the digest-pinned base answers "Found Vulkan: ... found components: glslc" and "Including Vulkan backend"; gate 1 (RADV GFX1151 visible to a rootless container off this base) is recorded in the verdict report.
This commit is contained in:
1 parent
7601edfe28
commit
25df9d05a2
5 files changed
+179
-38
No files matched your search
@@ -7,7 +7,7 @@
|
||||
# supervision, no retries, no logging of its own. PID 1 stays the server, so
|
||||
# `podman stop` and the unit's Restart=always drive it directly.
|
||||
#
|
||||
# THE FLAGS THAT ARE NOT HERE, and must stay absent:
|
||||
# THE FLAG THAT MUST STAY ABSENT:
|
||||
#
|
||||
# --threads / -t does not exist in this build. src/backend.h computes the
|
||||
# CPU thread count as std::thread::hardware_concurrency()/2
|
||||
@@ -18,13 +18,19 @@
|
||||
# placement — pinning is deliberately not done anywhere in
|
||||
# this stack. See the env file header for the same warning
|
||||
# aimed at the operator.
|
||||
# --device left at its default. Every GPU backend is compiled out of
|
||||
# the image, so the default device pick can only land on
|
||||
# CPU; naming a device would add a way to be wrong.
|
||||
#
|
||||
# Everything defaults to the shape the 2026-10-18 trial measured, so an empty
|
||||
# environment still boots the decided configuration: 1.7 B talker Q4_K_M with
|
||||
# the Q8_0 tokenizer, one request at a time, on the private bridge.
|
||||
# --device IS here, but conditionally: it reaches the binary only when
|
||||
# QWENTTS_DEVICE is non-empty, and empty is the CPU rollback — the binary then
|
||||
# takes its own default device, which is CPU because the CPU backend is always
|
||||
# built. One line in scripts/frank-tts-engine.env moves the endpoint between the
|
||||
# two shapes, with no rebuild. What must never reach argv is a device NAME the
|
||||
# loader does not enumerate: that is not refused by the argument parser, it dies
|
||||
# during device lookup with the loader's own error, so the journal reads as a
|
||||
# load that failed rather than a configuration refusal.
|
||||
#
|
||||
# Everything defaults to the shape this endpoint runs: 1.7 B talker Q4_K_M with
|
||||
# the Q8_0 tokenizer, one request at a time, on the private bridge, on the
|
||||
# gfx1151 iGPU. An empty environment still boots that shape.
|
||||
set -euo pipefail
|
||||
|
||||
# The binary path. Overridable for one reason: the bats seam runs a stub
|
||||
@@ -60,13 +66,33 @@ set -euo pipefail
|
||||
: "${QWENTTS_LANG:=auto}"
|
||||
|
||||
# One request at a time. The server-side queue is a serialised FIFO at this
|
||||
# setting, which is what keeps the measured per-voice timings true; batching
|
||||
# on a CPU build splits the same cores between two syntheses and the warm RTF
|
||||
# margin (0.55-0.67 against real time 1.0) disappears. The engine's own
|
||||
# default is also 1; it is written here because it is a decision, not an
|
||||
# accident.
|
||||
# setting, which is what keeps the per-voice timings predictable. A second
|
||||
# concurrent synthesis shares the device with the first — and on this box that
|
||||
# device is also carrying frank-dragoon — so it buys no throughput and it
|
||||
# doubles the wait for the caller behind it. The engine's own default is also 1;
|
||||
# written here because it is a decision, not an accident.
|
||||
: "${QWENTTS_MAX_BATCH:=1}"
|
||||
|
||||
# The compute device, NAMED and not numbered: `Vulkan0` is how ggml enumerates
|
||||
# the gfx1151 iGPU on this host. An index would follow whatever order the
|
||||
# loader's device list returns; a name says which device this endpoint means.
|
||||
# The container sees exactly one Vulkan device because the image carries the
|
||||
# Mesa ICD and no NVIDIA one, so the 3090 is not in the enumeration at all.
|
||||
#
|
||||
# EMPTY IS THE CPU ROLLBACK: a value that is set but EMPTY passes no `--device`,
|
||||
# the binary picks its own default, and the CPU backend is always built — which
|
||||
# is why moving this endpoint between the iGPU and the CPU is one line in
|
||||
# scripts/frank-tts-engine.env and not a rebuild.
|
||||
#
|
||||
# That is also why the default below is NOT `${QWENTTS_DEVICE:=Vulkan0}`: `:=`
|
||||
# fires on a set-but-empty variable as well as an unset one, so the rollback line
|
||||
# `QWENTTS_DEVICE=` would be rewritten back to Vulkan0 and the endpoint would sit
|
||||
# on the shared GPU with no way out short of a rebuild. Only UNSET takes the
|
||||
# default.
|
||||
if [[ ! -v QWENTTS_DEVICE ]]; then
|
||||
QWENTTS_DEVICE=Vulkan0
|
||||
fi
|
||||
|
||||
# Fail loudly BEFORE binding, so the journal says which artifact is missing
|
||||
# instead of the unit reporting a container that starts and dies. This is the
|
||||
# self-healing path: Restart=always re-runs it every RestartSec, and the same
|
||||
@@ -79,11 +105,20 @@ for artifact in "$QWENTTS_MODEL" "$QWENTTS_CODEC"; do
|
||||
fi
|
||||
done
|
||||
|
||||
exec "$QWENTTS_BIN" \
|
||||
--model "$QWENTTS_MODEL" \
|
||||
--codec "$QWENTTS_CODEC" \
|
||||
--alias "$QWENTTS_ALIAS" \
|
||||
--host "$QWENTTS_HOST" \
|
||||
--port "$QWENTTS_PORT" \
|
||||
--lang "$QWENTTS_LANG" \
|
||||
ARGS=(
|
||||
--model "$QWENTTS_MODEL"
|
||||
--codec "$QWENTTS_CODEC"
|
||||
--alias "$QWENTTS_ALIAS"
|
||||
--host "$QWENTTS_HOST"
|
||||
--port "$QWENTTS_PORT"
|
||||
--lang "$QWENTTS_LANG"
|
||||
--max-batch "$QWENTTS_MAX_BATCH"
|
||||
)
|
||||
|
||||
# Only when a device is named. An empty QWENTTS_DEVICE has to produce NO flag,
|
||||
# because "the rollback shape" is argv identical to the pre-iGPU endpoint.
|
||||
if [[ -n "$QWENTTS_DEVICE" ]]; then
|
||||
ARGS+=(--device "$QWENTTS_DEVICE")
|
||||
fi
|
||||
|
||||
exec "$QWENTTS_BIN" "${ARGS[@]}"
|
||||
Reference in new issue
Block a user