docs(frank-tts): gate 1 passes — RADV gives a container tenant the aperture, not the carve-out

vulkaninfo inside the engine's own base image (ubuntu:24.04 + mesa-vulkan-drivers
vulkan-tools, --device /dev/dri, rootless) reports Radeon 8060S Graphics (RADV
GFX1151) / driver radv / API 1.4.318 with two heaps: heap 0 host-visible 38.67 GiB
(budget 32.58) and heap 1 DEVICE_LOCAL 77.33 GiB (budget 65.16) with frank-dragoon
loaded. ggml's Vulkan backend asks for DEVICE_LOCAL, so the ~4 GiB the TTS engine
needs lands in the 65 GiB budget: memory is not the constraint, and the 2 GiB DRM
carve-out never enters the picture. Both heaps are GTT — the same host RAM, so the
budget is a ceiling on mappings, not free silicon memory.

Traps recorded: vulkaninfo --batch is not a flag in Ubuntu 24.04's vulkan-tools
(prints help, exits 1; use --text), and the container has no NVIDIA ICD, so Vulkan
device 0 happens to be the iGPU — select by name, not index.

Probe: /home/dank/spikes/npu-tts/vk_probe.sh on Dankclaw. Enumeration only; no
allocation, no compute, production untouched.
This commit is contained in:
Frank
2026-10-19 14:10:12 -07:00
parent 1e52e45fae
commit 1e6b39c382
+25 -13
View File
@@ -28,10 +28,11 @@ container carries `libggml-base.so.0.27.0` + `libggml-cpu.so.0.27.0` + `tts-serv
nothing else. So: `-DGGML_VULKAN=ON` + `--device Vulkan0`, plus `/dev/dri` passthrough in nothing else. So: `-DGGML_VULKAN=ON` + `--device Vulkan0`, plus `/dev/dri` passthrough in
the engine quadlet (the ComfyUI pattern: `AddDevice` + `GroupAdd=keep-groups` + a the engine quadlet (the ComfyUI pattern: `AddDevice` + `GroupAdd=keep-groups` + a
world-writable udev rule). The target shape is **co-tenancy** — the engine resident on the world-writable udev rule). The target shape is **co-tenancy** — the engine resident on the
gfx1151 *beside* `frank-dragoon`, not a handover — so §5 carries the census and the two gfx1151 *beside* `frank-dragoon`, not a handover. §5 carries the census and both gates:
gates that decide it: the heap RADV offers an APU tenant, and dragoon's TTFT with a the **heap gate is PASSED** — RADV's `DEVICE_LOCAL` heap is 77.33 GiB with 65.16 GiB of
synthesis in flight. Unproven, stated: whether qwentts.cpp's codec ops have Vulkan kernels budget while dragoon is loaded, so memory is not the constraint — which leaves the
for every op. **[host]** **[spike]** contention gate (dragoon's TTFT with a synthesis in flight) and one unproven piece:
whether qwentts.cpp's codec ops have Vulkan kernels for every op. **[host]** **[spike]**
Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs
only **12.5 codes/s** (§2), so this is latency headroom and freed cores, not reviving a only **12.5 codes/s** (§2), so this is latency headroom and freed cores, not reviving a
@@ -114,16 +115,27 @@ DRM "VRAM" carve-out|2,048 MiB total, 174 MiB used|what KFD/DRM calls dedicated
Host RAM|121 GiB total, 17 GiB available, 79 GiB buff/cache|**GTT pages are system pages**: an iGPU allocation is host RAM consumed, so the two tenants share one pool **[host]**| Host RAM|121 GiB total, 17 GiB available, 79 GiB buff/cache|**GTT pages are system pages**: an iGPU allocation is host RAM consumed, so the two tenants share one pool **[host]**|
TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that same pool **[host]**| TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that same pool **[host]**|
Two gates, in this order, before any deploy: **Two gates, in this order.**
1. **Heap gate (unmeasured — the number that decides everything else).** What heap 1. **Heap gate — PASSED.** RADV does **not** hand a container tenant the 2 GiB DRM
RADV offers a tenant on this APU is not known here: `mesa-vulkan-drivers 26.0.3` and carve-out. `vulkaninfo` inside the engine's own base image (`ubuntu:24.04` plus
`radeon_icd.json` are installed, `vulkan-tools` is not. Enumerate `mesa-vulkan-drivers vulkan-tools`, `--device /dev/dri`, rootless) sees
`VkPhysicalDeviceMemoryProperties` inside the container that will host the engine — `Radeon 8060S Graphics (RADV GFX1151)`, driver `radv`, API 1.4.318, and reports:
**before** loading a model. If the device-local heap is the 2 GiB carve-out rather than
the aperture, weights + KV + compute buffers fall back allocation-by-allocation and the |Heap|size|budget with dragoon loaded|flags|
iGPU win is gone. **[inferred]** — the fallback behavior is the risk, and the missing |---|---|---|---|
number is the gate. |0|38.67 GiB|32.58 GiB|host-visible, not device-local|
|1|**77.33 GiB**|**65.16 GiB**|`DEVICE_LOCAL`|
The heaps sum to the ~116 GiB aperture in Mesa's ⅓/⅔ APU split — the same RAM twice:
a Vulkan allocation is a GTT mapping, so a budget is a ceiling on mappings, not a pile
of free silicon memory. ggml's Vulkan backend asks for `DEVICE_LOCAL`, which is heap 1,
and the TTS engine needs roughly 4 GiB of it. **Memory stops being the question.** **[spike]**
Probe traps: `vulkaninfo --batch` is not a flag in this `vulkan-tools` (it prints the
help page and exits 1 — use `vulkaninfo --text`), and the container has **no NVIDIA
ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by
device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the
3090 slot.
2. **Contention gate.** `frank-dragoon` is the endpoint the operator session runs on, so the 2. **Contention gate.** `frank-dragoon` is the endpoint the operator session runs on, so the
honest measurement with both loaded is dragoon's own TTFT and decode with a synthesis in honest measurement with both loaded is dragoon's own TTFT and decode with a synthesis in
flight — one request each, no ladder. Expect compute-unit sharing rather than memory flight — one request each, no ladder. Expect compute-unit sharing rather than memory