command-center/agent/managed-skills/amd-apu-vulkan-tenant-onboarding/SKILL.md

name, description

name	description
amd-apu-vulkan-tenant-onboarding	Add a ggml/Vulkan inference tenant to an AMD APU (gfx1151 / Strix Halo iGPU) that already serves another engine: APU memory census, RADV heap gate, render-node resolution, build-side glslc/ICD deps, co-tenancy measurement.

Use when a ggml-based engine (llama.cpp, qwentts.cpp, anything built on ggml) must move from CPU to the iGPU of an AMD APU (gfx1151 / Strix Halo / Ryzen AI Max) that already serves another engine on the same card. Answers two questions only a booted host can: what gate the second tenant must clear, and which files the build needs to clear it.

1. The usual VRAM reasoning is wrong on an APU

/sys/class/drm/cardN/device/mem_info_vram_total reports a small carve-out (2 GiB class), and rocm-smi's "dedicated" figure is not the working set either. The real working set lives in GTT / system pages: /proc/meminfo → GTT and System pages. A second GPU tenant therefore consumes host RAM from the same pool the first tenant's weights and the OS use — no supplementary groups, no separate aperture to reserve.

• Price the tenant from a booted footprint, never from weights + KV + buffers: on an APU the compute buffers and any projector ride on GTT, which is host RAM; check free after the load, not before.
• Never subtract a "dedicated" figure from it to find headroom.
• Unheld fabric clock (pp_dpm_mech = auto) is a hazard only when the NPU joins too; a pure iGPU pair is unaffected — check it before adding a third tenant.
• Swap on a host with GPU tenants protects co-tenants; paging a KV pool is worse than the OOM it prevents.

2. The gate that decides feasibility: the RADV heap a container sees

rocm-smi and free cannot see this number; only a container off the engine's own base image can:

# one-off container, no model, no load, no compute — safe beside production
podman run --rm --device /dev/dri --group-add keep-groups <engine-base> bash -c \
  'apt-get update -qq >/dev/null 2>&1; apt-get install -y -qq --no-install-recommends mesa-vulkan-drivers vulkan-tools glslang-tools >/dev/null 2>&1; vulkaninfo --text'

Read VkPhysicalDeviceMemoryProperties → the DEVICE_LOCAL heap size. On a hybrid Strix Halo box Mesa splits the aperture (~1/3 : 2/3), so DEVICE_LOCAL is the budget the ggml tool will actually allocate against — measured 77.33 GiB with the first tenant loaded, against a 2 GiB DRM carve-out. Two traps, both measured:

• vulkaninfo --batch does not exist in Ubuntu 24.04's vulkan-tools (prints help, exits 1). Every vulkaninfo parse needs --text.
• The container may have no NVIDIA ICD, so Vulkan index 0 is the iGPU — but do not trust ordering by index: --text and read deviceName + driverID (RADV_GFX1151 vs NVIDIA). A hand-picked /dev/dri/renderDN is worse than useless on a hybrid host: renderD128 there is the NVIDIA card and renderD129 the amdgpu iGPU; resolve the node with readlink -f /sys/class/drm/renderD*/device/driver and pass /dev/dri wholesale, letting the driver pick by name.
• A rootless container that drops its supplementary groups enumerates nothing. --group-add keep-groups (quadlet: GroupAdd=keep-groups) plus the host udev rule that makes the render nodes group-writable is the whole requirement; --device /dev/dri alone is not enough.

3. Build-side deps: the configure fails before the device ever matters

ggml's src/ggml-vulkan/CMakeLists.txt requires, in order: find_package(Vulkan COMPONENTS glslc REQUIRED) then find_package(SPIRV-Headers CONFIG REQUIRED), and it compiles its shaders with glslc, never glslangValidator. On Ubuntu 24.04 (measured against the digest-pinned ubuntu:24.04 base, 2026-10-19):

• glslang-tools ships glslangValidator and NOT glslc → command -v glslc empty, find_package(Vulkan … glslc REQUIRED) fails, and the Vulkan backend silently never exists.
• The package is the standalone glslc (universe, 2023.8-1build1). Working build-stage set: libvulkan-dev glslc spirv-headers — libvulkan-dev provides vulkan/vulkan.h + the VulkanHeaders/VulkanLoader CMake configs find_package(Vulkan) resolves to, spirv-headers ships /usr/share/cmake/SPIRV-Headers/SPIRV-HeadersConfig.cmake. A configure probe in the pinned base prints Found Vulkan: … found components: glslc then Including Vulkan backend.
• The archive's glslc is old. 2023.8 compiles GL_KHR_cooperative_matrix (the path ggml's vectored quantized matmul needs) but reports cooperative_matrix2, integer_dot_product, bfloat16, float_e2m1/e4m3 as unsupported, so those shader families are compiled out. If a Vulkan prefill reads as inexplicably slow, suspect the shader compiler before the device; the fix is a pinned glslang build passed as -DVulkan_GLSLC_EXECUTABLE=<path>, not a different runtime.
• Runtime stage needs the loader and an ICD: libvulkan1 + mesa-vulkan-drivers. Loader-without-ICD enumerates zero devices, and ggml-based servers die in device lookup — often after reading the weights, so the journal reads like a model fault.
• Ship no NVIDIA ICD on purpose: one Vulkan device in the enumeration is what makes the name Vulkan0 safe to write into config. Prefer the name over an index in every flag.

4. Prove enumeration without allocating anything

Do not boot a scratch second engine beside production to ask which devices exist — a second model load is a real GPU tenant, and fleet law stops that. Instead run the binary with a device name that cannot exist and read its error; it names what it enumerated, with no allocation and no model resident:

podman run --rm --device /dev/dri --group-add keep-groups --entrypoint /app/tts-server <image> \
  --model /dev/null --codec /dev/null --device NoSuchDevice 2>&1 | tail -20

5. The rollback must be a flag, not a rebuild

Put the device in the env file as a value the entrypoint passes only when set, and make set-but-empty mean "pass no flag at all" — the binary's own default (CPU, always built) is the rollback. : "${VAR:=default}" is the WRONG form: := fires on an empty value too, so the documented rollback line VAR= is silently rewritten back to the GPU and the only way out is a rebuild. Use:

if [[ ! -v QWENTTS_DEVICE ]]; then QWENTTS_DEVICE=Vulkan0; fi

(podman's --env-file passes VAR= through as set-and-empty, and source does the same, so both readers agree.) Pin it in a test: empty device ⇒ the flag is absent from argv.

6. Co-tenancy measurement (only after the tenant is live)

The gate is the first tenant's performance with a real request from the second in flight — not the second tenant's own speed. And price the tenant from a booted footprint, never from weights+KV+buffers: on an APU the compute buffers and any projector ride on GTT, which is host RAM; check free after the load, not before.