command-center/agent/managed-skills/amd-apu-vulkan-tenant-onboarding/SKILL.md

name, description

name	description
amd-apu-vulkan-tenant-onboarding	Add a ggml/Vulkan inference tenant to an AMD APU (gfx1151 / Strix Halo iGPU) that already serves another engine: APU memory census, the measured RADV heap gate, render-node resolution on hybrid hosts, build/runtime dependency surface, co-tenancy measurement.

Add a ggml/Vulkan inference tenant to an AMD APU (gfx1151 / Strix Halo iGPU) that already serves another engine: APU memory census, the RADV heap gate, the build/runtime dependency surface, and co-tenancy measurement.

Use when a ggml-based engine (llama.cpp, qwentts.cpp, anything built on ggml) must move from CPU to the iGPU of an AMD APU (gfx115x / Strix Halo / Ryzen AI) that already serves a production tenant. Answers: what budget actually exists, what gate decides feasibility before a model loads, and exactly which files a Vulkan build needs.

Why the usual VRAM reasoning is wrong on an APU

/sys/class/drm/card*/device/mem_info_vram_total reports a small carve-out (2 GiB class), while the real working set lives in GTT — and GTT pages are system pages. A second GPU tenant therefore consumes host RAM from the same pool the first tenant and the OS draw from; there is no second memory to size against. The aperture is almost never the constraint; available RAM and shader time are.

Census (read-only; run before proposing anything)

for c in /sys/class/drm/card*; do
  v=$(cat $c/device/mem_info_vram_used 2>/dev/null); g=$(cat $c/device/mem_info_gtt_used 2>/dev/null)
  [ -n "$v$g" ] && echo "$(basename $c) vram_used=$((v/1048576))MiB gtt_used=$((g/1048576))MiB vram_total=$(($(cat $c/device/mem_info_vram_total)/1048576))MiB"
done
cat /sys/class/drm/card*/device/mem_info_gtt_total   # the aperture, in bytes
free -g                                              # available is the number that bites

Resolve the render node by driver, never by number — on a hybrid host (dGPU + APU) renderD128 is often the other vendor's node:

for r in /sys/class/drm/renderD*; do
  echo "$(basename $r) $(basename $(readlink -f $r/device/driver))"
done        # gfx1151 example on a box with an RTX 3090: renderD128 = nvidia, renderD129 = amdgpu

Also check what the userspace offers before promising a backend:

dpkg -l | grep -iE "mesa-vulkan|vulkan-tools"        # ICD present? probe installed?
ls /usr/share/vulkan/icd.d/                          # radeon_icd.json = RADV available
ls <image>/usr/share/vulkan/icd.d/                   # the ICD must exist INSIDE the container

Gate 1 — the heap, before any model loads

Ask what heap the Vulkan driver hands a tenant on this APU, in the same container image that will host the engine, with /dev/dri passed through:

podman run --rm --device /dev/dri -v <dir>:/probe:rw <engine-build-base> bash -lc '
  apt-get update -qq && apt-get install -y -qq --no-install-recommends mesa-vulkan-drivers vulkan-tools
  vulkaninfo --text > /probe/full.txt'          # NOT --batch: that flag does not exist and prints help, rc=1
awk "/memoryHeaps: count/,/memoryTypes: count/" /probe/full.txt   # sizes + budgets + DEVICE_LOCAL flag

Measured on gfx1151 (Strix Halo, 116 GiB aperture) with a production ROCm tenant already loaded: Radeon 8060S Graphics (RADV GFX1151), driver radv, API 1.4.318, heap 0 host-visible 38.67 GiB (budget 32.58) and heap 1 DEVICE_LOCAL 77.33 GiB (budget 65.16). Mesa splits the aperture roughly ⅓/⅔ on APUs, so DEVICE_LOCAL is the aperture, not the 2 GiB carve-out — the "if the heap is the carve-out, give up" branch does not trigger on this silicon. ggml asks for DEVICE_LOCAL, so a few-GiB engine lands inside a tens-of-GiB budget and memory stops being the question. Both heaps are GTT: the same host RAM, so a budget is a ceiling on mappings, not free silicon memory — free still decides whether it can be touched.

Two probe traps: vulkaninfo --batch is not a flag in Ubuntu 24.04's vulkan-tools (it prints the help page and exits 1), and a slim container has no nvidia_icd.json, so Vulkan device 0 happens to be the iGPU there — an image that grows an NVIDIA ICD silently reorders the list. Select by device name, never by index.

This is enumeration: no allocation, no compute — but it opens the render node on a production card, so it is still a decision for the operator, not a routine probe.

Gate 2 — contention, measured on the co-tenant

If the target shape is co-tenancy (both resident), the honest measurement is the existing tenant's TTFT and decode with the new tenant's work in flight — one request each, no ladder. Stopping the co-tenant is for benchmarks, not for a deployment shape. Expect compute-unit sharing, not memory pressure.

The dependency surface people miss

1. Build stage needs more than the flag. ggml compiles its Vulkan shaders at build time: -DGGML_VULKAN=ON alone fails at configure without libvulkan-dev and glslang-dev (or glslang-tools). A CPU-only image's build stage typically installs only cmake g++ git make.
2. The runtime stage needs an ICD, not just a device node. mesa-vulkan-drivers must be installed in the image; /dev/dri without radeon_icd.json in the container buys nothing.
3. Rootless/quadlet device access: AddDevice=/dev/dri (or the iGPU's own node — see the census: it is not necessarily renderD128) + GroupAdd=keep-groups, and the host needs the world-writable udev rule (KERNEL=="renderD*", MODE="0666") — systemd user-unit spawns carry no supplementary groups, so the render group never arrives.
4. Runtime selection is a device flag on the engine (--device Vulkan0 for ggml-based tools), not an env var — and confirm which physical device that index resolves to after the image gains its ICD.

Laws

• Price the tenant from a booted footprint, never from weights + KV + buffers: on an APU the compute buffers and any projector ride on GTT, which is host RAM — check free after the load, not before.
• Never treat the vram_total carve-out as capacity, and never subtract a "dedicated" figure from it to find headroom.
• Don't reason from nvidia-smi-style intuition: on this platform the useful numbers are mem_info_gtt_used, free, and RADV's heap budgets.
• Unheld fabric clock (pp_dpm_fclk = auto) is a hazard only when the NPU joins too; a pure iGPU pair is unaffected — check it before adding a third tenant.
• Swap on a host with GPU tenants protects co-tenants; paging a KV pool is worse than the OOM it prevents.