command-center/agent/managed-skills/halogen-npu-feasibility-census/SKILL.md

name, description

name	description
halogen-npu-feasibility-census	Census whether halogen-flash-server's NPU small-model path can run on a Strix Halo host: image gate, amdxdna/XRT presence, IOMMU and fabric-clock blockers, enablement deltas.

Use when asked whether halogen/halogen-flash-server can use the XDNA2 NPU (embeddings, rerank, moderation/guard, decider, qwen3.5-2b generation, flux2-klein-4b images, qwen3-asr-*) on a Ryzen AI Max / Strix Halo host, or before proposing to enable it.

What the feature is (not a speed knob)

The Flash trunk stays on the iGPU. HALOGEN_NPU_MODELS=<names> starts a second engine in the SAME container and on the SAME API port; the request's model field picks the executor. Unset → server identical to before. Each NPU model is a precompiled AIE-ML program (devices/devices.hnpm, per-partition devices/u<N>.elf, devices/consts.hnpb, <id>.hnpw weights, tokenizer/tokenizer.json), loaded via the HOST's XRT → kernel amdxdna → /dev/accel/accel0. One pass at a time; HALOGEN_NPU_QUEUE 64, overflow queues server-side, then 503 engine_busy. Entry entrypoint.sh validates every file's bytes + sha256 against /opt/halogen/npu/models.txt before anything starts. NPU models run only in single-container "all" mode, NOT the engine/api split.

Census (all read-only, run from the operator box)

ssh <host> '
# 1. engine gate + catalog in the pinned image
podman exec <ctr> bash -c "grep -rlo HALOGEN_NPU_MODELS /opt/halogen /usr/local/bin /entrypoint.sh 2>/dev/null | head"
podman exec <ctr> cat /opt/halogen/npu/models.txt | head -40
# 2. is NPU actually wired today? (expect NO env, NO device, NO /dev/accel inside)
podman inspect <ctr> --format "{{json .HostConfig.Devices}}"; podman exec <ctr> ls /dev/accel
# 3. kernel driver + firmware
lsmod | grep -i amdxdna; sudo dmesg | grep -iE "xdna|npu" | tail -5; ls -l /dev/accel
# 4. XRT (distro packages are fine; vendor mounts /opt/xilinx/xrt)
dpkg -l | grep -iE "xrt|xdna"; ls /usr/lib/x86_64-linux-gnu/libxrt* /usr/lib/libxrt* 2>/dev/null
# 5. BLOCKER 1: IOMMU
cat /proc/cmdline; ls /sys/kernel/iommu_groups | wc -l
# 6. BLOCKER 2: fabric clock held?
for c in /sys/class/drm/card*/device; do echo "== $c"; cat $c/power_dpm_force_performance_level 2>&1; cat $c/pp_dpm_fclk 2>&1 | tr "\n" " "; echo; done'

Docs (raw fetch, authoritative): docs/NPU.md, docs/npu-small-models.md, deploy/host/halogen-fabric-clock under peonist-ai/halogen-flash-server on GitHub. Tag list: podman search --list-tags ghcr.io/peonist-ai/halogen-flash-server.

The two blockers and the enablement delta

• IOMMU must be on. amd_iommu=off turns the NPU off (no PASID/SVA); iommu=pt keeps it. Needs a grub edit + reboot. iommu_groups == 0 ⇒ off.
• Fabric clock must be pinned at the top pp_dpm_fclk level (or power_dpm_force_performance_level=high); auto = unheld. GPU+NPU concurrently HANG Strix Halo machines and corrupt NPU results while that clock moves. Fix: root container with -v /sys:/host/sys (server holds it itself), or install deploy/host/halogen-fabric-clock + unit (~1 W). If it can't hold, the server FAILS CLOSED and prints the commands.
• Container delta: --device /dev/accel/accel0 + HALOGEN_NPU_MODELS; --ulimit memlock=-1:-1 and --group-add keep-groups usually already present. Distro XRT: mount each of libxrt_coreutil.so.2, libxrt_core.so.2, libxrt_driver_xdna.so.2 TWICE (once at /opt/xilinx/xrt/lib/, once at its own path) — never mount an /opt/xilinx/xrt holding only symlinks.

Costs to price before proposing

• Upstream A/B: IOMMU translation ON costs 13–16 % prefill (460 → 385 tok/s at 2,048 tokens, Ryzen AI Max+ 395); decode unaffected. Whether iommu=pt specifically costs that on a given box is a local measurement, not a published fact — run the image's own sweep after the reboot.
• A few GB host RAM per NPU model (+8 GB FLUX, +1.6/4 GB ASR), and trunk decode drops while the NPU works (shared bus + power).

Laws / traps

• Never probe the NPU beside the live service: stop + mask first (fleet benchmarking policy); this host class has a precedent of a concurrent GPU experiment needing a power cycle. Distinguish off vs wedged with ip neigh show <ip> from a peer.
• /dev/accel/accel0 existing and amdxdna probing cleanly does NOT mean the NPU works — with IOMMU off the failure is at context/PASID creation, not at the device node. Do not trust older notes claiming the node itself is missing.
• dmesg lines like Failed to alloc dev bo memory, ret -28 while large NPU models take turns at NPU memory are benign (vendor-documented); the server recovers.