docs(frank-tts): gate 1 passes — RADV gives a container tenant the aperture, not the carve-out
vulkaninfo inside the engine's own base image (ubuntu:24.04 + mesa-vulkan-drivers vulkan-tools, --device /dev/dri, rootless) reports Radeon 8060S Graphics (RADV GFX1151) / driver radv / API 1.4.318 with two heaps: heap 0 host-visible 38.67 GiB (budget 32.58) and heap 1 DEVICE_LOCAL 77.33 GiB (budget 65.16) with frank-dragoon loaded. ggml's Vulkan backend asks for DEVICE_LOCAL, so the ~4 GiB the TTS engine needs lands in the 65 GiB budget: memory is not the constraint, and the 2 GiB DRM carve-out never enters the picture. Both heaps are GTT — the same host RAM, so the budget is a ceiling on mappings, not free silicon memory. Traps recorded: vulkaninfo --batch is not a flag in Ubuntu 24.04's vulkan-tools (prints help, exits 1; use --text), and the container has no NVIDIA ICD, so Vulkan device 0 happens to be the iGPU — select by name, not index. Probe: /home/dank/spikes/npu-tts/vk_probe.sh on Dankclaw. Enumeration only; no allocation, no compute, production untouched.
This commit is contained in:
@@ -28,10 +28,11 @@ container carries `libggml-base.so.0.27.0` + `libggml-cpu.so.0.27.0` + `tts-serv
|
||||
nothing else. So: `-DGGML_VULKAN=ON` + `--device Vulkan0`, plus `/dev/dri` passthrough in
|
||||
the engine quadlet (the ComfyUI pattern: `AddDevice` + `GroupAdd=keep-groups` + a
|
||||
world-writable udev rule). The target shape is **co-tenancy** — the engine resident on the
|
||||
gfx1151 *beside* `frank-dragoon`, not a handover — so §5 carries the census and the two
|
||||
gates that decide it: the heap RADV offers an APU tenant, and dragoon's TTFT with a
|
||||
synthesis in flight. Unproven, stated: whether qwentts.cpp's codec ops have Vulkan kernels
|
||||
for every op. **[host]** **[spike]**
|
||||
gfx1151 *beside* `frank-dragoon`, not a handover. §5 carries the census and both gates:
|
||||
the **heap gate is PASSED** — RADV's `DEVICE_LOCAL` heap is 77.33 GiB with 65.16 GiB of
|
||||
budget while dragoon is loaded, so memory is not the constraint — which leaves the
|
||||
contention gate (dragoon's TTFT with a synthesis in flight) and one unproven piece:
|
||||
whether qwentts.cpp's codec ops have Vulkan kernels for every op. **[host]** **[spike]**
|
||||
|
||||
Why the iGPU is worth doing at all, given CPU already works: the deployed payload needs
|
||||
only **12.5 codes/s** (§2), so this is latency headroom and freed cores, not reviving a
|
||||
@@ -114,16 +115,27 @@ DRM "VRAM" carve-out|2,048 MiB total, 174 MiB used|what KFD/DRM calls dedicated
|
||||
Host RAM|121 GiB total, 17 GiB available, 79 GiB buff/cache|**GTT pages are system pages**: an iGPU allocation is host RAM consumed, so the two tenants share one pool **[host]**|
|
||||
TTS engine today|3.84 GiB RSS, CPU|its device-resident replacement lands in that same pool **[host]**|
|
||||
|
||||
Two gates, in this order, before any deploy:
|
||||
**Two gates, in this order.**
|
||||
|
||||
1. **Heap gate (unmeasured — the number that decides everything else).** What heap
|
||||
RADV offers a tenant on this APU is not known here: `mesa-vulkan-drivers 26.0.3` and
|
||||
`radeon_icd.json` are installed, `vulkan-tools` is not. Enumerate
|
||||
`VkPhysicalDeviceMemoryProperties` inside the container that will host the engine —
|
||||
**before** loading a model. If the device-local heap is the 2 GiB carve-out rather than
|
||||
the aperture, weights + KV + compute buffers fall back allocation-by-allocation and the
|
||||
iGPU win is gone. **[inferred]** — the fallback behavior is the risk, and the missing
|
||||
number is the gate.
|
||||
1. **Heap gate — PASSED.** RADV does **not** hand a container tenant the 2 GiB DRM
|
||||
carve-out. `vulkaninfo` inside the engine's own base image (`ubuntu:24.04` plus
|
||||
`mesa-vulkan-drivers vulkan-tools`, `--device /dev/dri`, rootless) sees
|
||||
`Radeon 8060S Graphics (RADV GFX1151)`, driver `radv`, API 1.4.318, and reports:
|
||||
|
||||
|Heap|size|budget with dragoon loaded|flags|
|
||||
|---|---|---|---|
|
||||
|0|38.67 GiB|32.58 GiB|host-visible, not device-local|
|
||||
|1|**77.33 GiB**|**65.16 GiB**|`DEVICE_LOCAL`|
|
||||
|
||||
The heaps sum to the ~116 GiB aperture in Mesa's ⅓/⅔ APU split — the same RAM twice:
|
||||
a Vulkan allocation is a GTT mapping, so a budget is a ceiling on mappings, not a pile
|
||||
of free silicon memory. ggml's Vulkan backend asks for `DEVICE_LOCAL`, which is heap 1,
|
||||
and the TTS engine needs roughly 4 GiB of it. **Memory stops being the question.** **[spike]**
|
||||
Probe traps: `vulkaninfo --batch` is not a flag in this `vulkan-tools` (it prints the
|
||||
help page and exits 1 — use `vulkaninfo --text`), and the container has **no NVIDIA
|
||||
ICD**, so Vulkan device 0 is the iGPU here. Do not trust that ordering — select by
|
||||
device *name*, or an image that grows an `nvidia_icd.json` silently points TTS at the
|
||||
3090 slot.
|
||||
2. **Contention gate.** `frank-dragoon` is the endpoint the operator session runs on, so the
|
||||
honest measurement with both loaded is dragoon's own TTFT and decode with a synthesis in
|
||||
flight — one request each, no ladder. Expect compute-unit sharing rather than memory
|
||||
|
||||
Reference in New Issue
Block a user