docs(frank-tts): record the iGPU cutover, close the NPU question
The cutover PR merged the runtime; this carries the surface that has to be true afterwards. - Engine page: the shape is now `--device Vulkan0` on the gfx1151 as a co-tenant beside frank-dragoon; the registry/thread/streaming laws gain a fourth — the device law — whose numbers stay in the verdict report, and the deploy record gets the cutover steps including the `glslc` configure gate. The CPU-era arm/rollback prose keeps the qualitative argument and drops the dead numbers per the purpose test. - Endpoint page: "Where the engine computes" now says iGPU + co-tenancy, and states the NPU verdict in one paragraph instead of a research tour. - Quadlet comment: it claimed supplementary groups are what grant render access. On this host the udev rule world-writes the render nodes and the container's `id` shows no groups at all, so keep-groups is belt-and-braces; the RSS note now matches the device shape instead of the CPU trial. - Verdict report: gate 2 (contention) recorded as measured after the cutover, and the build-stage prediction corrected — the package is `glslc`, not `libvulkan-dev glslang-dev`. - index.md regenerated (one description changed).
This commit is contained in:
@@ -136,25 +136,36 @@ TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that
|
||||
ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by
|
||||
device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the
|
||||
3090 slot.
|
||||
2. **Contention gate.** `frank-dragoon` is the endpoint the operator session runs on, so the
|
||||
honest measurement with both loaded is dragoon's own TTFT and decode with a synthesis in
|
||||
flight — one request each, no ladder. Expect compute-unit sharing rather than memory
|
||||
pressure: the aperture has room, the shaders do not.
|
||||
2. **Contention gate — PASSED, measured 2026-10-19 after the cutover.** With a 16.9 s
|
||||
synthesis occupying the device, `frank-dragoon` answered a streamed 124-token request at
|
||||
**17 ms to first token and ~35 tok/s decode**, against 17 ms and ~33 tok/s for the same
|
||||
request with the card idle. Compute-unit sharing showed up as nothing. One request per
|
||||
arm, no ladder: these are the gate's evidence, not a throughput record, and the engine's
|
||||
own shape lives in [[Frank TTS Engine]].
|
||||
|
||||
**Change surface** — the whole edit, so nobody re-derives it:
|
||||
**Change surface — as landed** (PR #34, `6248f02`; the CPU shape stays reachable as the
|
||||
rollback):
|
||||
|
||||
1. `Containerfile.engine` build stage: add `libvulkan-dev glslang-dev`. It installs only
|
||||
`ca-certificates cmake g++ git make` today, and ggml's Vulkan backend compiles its
|
||||
shaders at build time — the flag alone fails at configure.
|
||||
2. `-DGGML_VULKAN=OFF` → `ON` in the same cmake block; the flag list is written out
|
||||
explicitly, so this is a one-word diff.
|
||||
3. Runtime stage: add `mesa-vulkan-drivers`. Without a Vulkan **ICD inside the image** the
|
||||
device node buys nothing — there is no `radeon_icd.json` in the container today.
|
||||
4. `frank-tts-engine.container`: `AddDevice=/dev/dri` + `GroupAdd=keep-groups` — **not**
|
||||
`renderD128`. On this hybrid box that node is the NVIDIA card; the iGPU is
|
||||
`renderD129` (`readlink -f /sys/class/drm/renderD129/device/driver` → `amdgpu`).
|
||||
5. Engine argv: `--device Vulkan0` — qwentts.cpp's tools take `--device <name>` at the
|
||||
pinned commit.
|
||||
1. `Containerfile.engine` build stage: add **`libvulkan-dev glslc spirv-headers`**. The
|
||||
prediction above (`libvulkan-dev glslang-dev`) is wrong on two counts and the configure
|
||||
says so: ggml looks for `glslc`, which Ubuntu's `glslang-tools` does **not** ship (it
|
||||
ships `glslangValidator`), and `find_package(SPIRV-Headers CONFIG REQUIRED)` needs the
|
||||
distro's `spirv-headers`, which does carry the CMake config. `glslc` is its own source
|
||||
package on noble, at 2023.8 — old enough that only `GL_KHR_cooperative_matrix` compiles,
|
||||
so the coopmat2 / integer-dot / bf16 / micro-float shader families are built out. Suspect
|
||||
that compiler before the device if a Vulkan prefill ever reads as slow.
|
||||
2. `-DGGML_VULKAN=OFF` → `ON`: one word in the written-out flag list.
|
||||
3. Runtime stage: `mesa-vulkan-drivers` **and** `libvulkan1` — the ICD and its loader. An
|
||||
image with a loader and no ICD enumerates zero devices.
|
||||
4. `frank-tts-engine.container`: `AddDevice=/dev/dri` + `GroupAdd=keep-groups` — the
|
||||
DIRECTORY, never the node: on this hybrid box `renderD128` is the NVIDIA card and
|
||||
`renderD129` the amdgpu iGPU (`readlink -f /sys/class/drm/renderD129/device/driver` →
|
||||
`amdgpu`). On this host the udev rule already makes the nodes world-writable, so the
|
||||
group is belt-and-braces, not the gate.
|
||||
5. Engine argv: `--device Vulkan0`, reached through `QWENTTS_DEVICE` in
|
||||
`scripts/frank-tts-engine.env`. **An empty value passes no flag at all** — that is the
|
||||
CPU rollback, and it is why the entrypoint's default is an `-v` test rather than `:=`,
|
||||
which also fires on a set-but-empty value and would silently undo the rollback.
|
||||
|
||||
Gate 1 recipe, already run and **PASSED**: `mesa-vulkan-drivers` + `vulkan-tools` in a
|
||||
throwaway container off the same base, `--device /dev/dri`, then `vulkaninfo --text` and read
|
||||
|
||||
Reference in New Issue
Block a user