From 1e6b39c3827d4596cfa0db515114eb005d5eb94c Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 19 Oct 2026 14:10:12 -0700 Subject: [PATCH] =?UTF-8?q?docs(frank-tts):=20gate=201=20passes=20?= =?UTF-8?q?=E2=80=94=20RADV=20gives=20a=20container=20tenant=20the=20apert?= =?UTF-8?q?ure,=20not=20the=20carve-out?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit vulkaninfo inside the engine's own base image (ubuntu:24.04 + mesa-vulkan-drivers vulkan-tools, --device /dev/dri, rootless) reports Radeon 8060S Graphics (RADV GFX1151) / driver radv / API 1.4.318 with two heaps: heap 0 host-visible 38.67 GiB (budget 32.58) and heap 1 DEVICE_LOCAL 77.33 GiB (budget 65.16) with frank-dragoon loaded. ggml's Vulkan backend asks for DEVICE_LOCAL, so the ~4 GiB the TTS engine needs lands in the 65 GiB budget: memory is not the constraint, and the 2 GiB DRM carve-out never enters the picture. Both heaps are GTT — the same host RAM, so the budget is a ceiling on mappings, not free silicon memory. Traps recorded: vulkaninfo --batch is not a flag in Ubuntu 24.04's vulkan-tools (prints help, exits 1; use --text), and the container has no NVIDIA ICD, so Vulkan device 0 happens to be the iGPU — select by name, not index. Probe: /home/dank/spikes/npu-tts/vk_probe.sh on Dankclaw. Enumeration only; no allocation, no compute, production untouched. --- report/tts-npu-spike-verdict-2026-10-19.md | 38 ++++++++++++++-------- 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/report/tts-npu-spike-verdict-2026-10-19.md b/report/tts-npu-spike-verdict-2026-10-19.md index e5e5f7d..8093397 100644 --- a/report/tts-npu-spike-verdict-2026-10-19.md +++ b/report/tts-npu-spike-verdict-2026-10-19.md @@ -28,10 +28,11 @@ container carries `libggml-base.so.0.27.0` + `libggml-cpu.so.0.27.0` + `tts-serv nothing else. So: `-DGGML_VULKAN=ON` + `--device Vulkan0`, plus `/dev/dri` passthrough in the engine quadlet (the ComfyUI pattern: `AddDevice` + `GroupAdd=keep-groups` + a world-writable udev rule). The target shape is **co-tenancy** — the engine resident on the -gfx1151 *beside* `frank-dragoon`, not a handover — so §5 carries the census and the two -gates that decide it: the heap RADV offers an APU tenant, and dragoon's TTFT with a -synthesis in flight. Unproven, stated: whether qwentts.cpp's codec ops have Vulkan kernels -for every op. **[host]** **[spike]** +gfx1151 *beside* `frank-dragoon`, not a handover. §5 carries the census and both gates: +the **heap gate is PASSED** — RADV's `DEVICE_LOCAL` heap is 77.33 GiB with 65.16 GiB of +budget while dragoon is loaded, so memory is not the constraint — which leaves the +contention gate (dragoon's TTFT with a synthesis in flight) and one unproven piece: +whether qwentts.cpp's codec ops have Vulkan kernels for every op. **[host]** **[spike]** Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs only **12.5 codes/s** (§2), so this is latency headroom and freed cores, not reviving a @@ -114,16 +115,27 @@ DRM "VRAM" carve-out|2,048 MiB total, 174 MiB used|what KFD/DRM calls dedicated Host RAM|121 GiB total, 17 GiB available, 79 GiB buff/cache|**GTT pages are system pages**: an iGPU allocation is host RAM consumed, so the two tenants share one pool **[host]**| TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that same pool **[host]**| -Two gates, in this order, before any deploy: +**Two gates, in this order.** -1. **Heap gate (unmeasured — the number that decides everything else).** What heap - RADV offers a tenant on this APU is not known here: `mesa-vulkan-drivers 26.0.3` and - `radeon_icd.json` are installed, `vulkan-tools` is not. Enumerate - `VkPhysicalDeviceMemoryProperties` inside the container that will host the engine — - **before** loading a model. If the device-local heap is the 2 GiB carve-out rather than - the aperture, weights + KV + compute buffers fall back allocation-by-allocation and the - iGPU win is gone. **[inferred]** — the fallback behavior is the risk, and the missing - number is the gate. +1. **Heap gate — PASSED.** RADV does **not** hand a container tenant the 2 GiB DRM + carve-out. `vulkaninfo` inside the engine's own base image (`ubuntu:24.04` plus + `mesa-vulkan-drivers vulkan-tools`, `--device /dev/dri`, rootless) sees + `Radeon 8060S Graphics (RADV GFX1151)`, driver `radv`, API 1.4.318, and reports: + + |Heap|size|budget with dragoon loaded|flags| + |---|---|---|---| + |0|38.67 GiB|32.58 GiB|host-visible, not device-local| + |1|**77.33 GiB**|**65.16 GiB**|`DEVICE_LOCAL`| + + The heaps sum to the ~116 GiB aperture in Mesa's ⅓/⅔ APU split — the same RAM twice: + a Vulkan allocation is a GTT mapping, so a budget is a ceiling on mappings, not a pile + of free silicon memory. ggml's Vulkan backend asks for `DEVICE_LOCAL`, which is heap 1, + and the TTS engine needs roughly 4 GiB of it. **Memory stops being the question.** **[spike]** + Probe traps: `vulkaninfo --batch` is not a flag in this `vulkan-tools` (it prints the + help page and exits 1 — use `vulkaninfo --text`), and the container has **no NVIDIA + ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by + device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the + 3090 slot. 2. **Contention gate.** `frank-dragoon` is the endpoint the operator session runs on, so the honest measurement with both loaded is dragoon's own TTFT and decode with a synthesis in flight — one request each, no ladder. Expect compute-unit sharing rather than memory