diff --git a/AGENTS.md b/AGENTS.md index 333c829..c7dc0e4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1108,9 +1108,15 @@ linked runbooks. - SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its ports off the gluetun service. - `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon, - 2026-09-11) permanently retired the XDNA2 NPU: the driver's SVA/PASID bind - needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). Revert path - in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`. + 2026-09-11) retires the XDNA2 NPU: the driver's SVA/PASID bind needs an + enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). That trade is on an + expiring path: **AMD PerfOpt** (iommu tree 2026-09-24, Linux 7.4) stops + iGPU GPA→SPA translation while the IOMMU stays enabled for every other + device, so the next boot-line window is `iommu=pt` with the NPU usable — + measured only 2–4 % residual on a Strix Halo desktop (18–23 % on Strix + Point laptops). Not installable here yet: Ubuntu 26.04 caps at + `7.0.0-38`. Full research in + `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`. - Rootless Quadlet containers run with `--replace --rm`: `podman logs` covers only the CURRENT instance; history across recreations lives in `journalctl --user -u .service`. Read rootless storage WITHOUT sudo diff --git a/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md b/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md index 2af7580..9eedb98 100644 --- a/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md +++ b/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md @@ -89,6 +89,70 @@ with the archived unit (`.AGENTTRASH/frank-npu.service.flm-npu`) if the NPU endpoint is wanted back. The IOMMU change is independent of the halogen deployment — reverting one does not require reverting the other. +## 2026-10-18 research: the penalty now has a hardware fix (AMD PerfOpt) + +Asked and answered — **the decision changes, the host does not.** Nothing was +changed on Dankclaw today; this section records why the 13–16 % figure is on an +expiring path. + +**Mechanism.** AMD IOMMU **PerfOpt**, spec-defined in the AMD I/O Virtualization +Technology (IOMMU) spec §3.4.9: MMIO `0x016C` bit 13 `PERF_OPT_EN`, gated by +extended-feature bit 45 `FEATURE_PERF_OPT`. Armed, it stops GPA→SPA translation +for the *privileged integrated GPU* — only the IR/IW permission bits are +enforced — while the IOMMU stays fully enabled for every other device, the NPU +included. Driver: `drivers/iommu/amd/{iommu.c,amd_iommu_types.h}`. The only +consumer is **amdgpu**: `amdgpu_device_use_perfopt()` requires `AMD_IS_APU` **and +the identity domain**, and the shared per-IOMMU bit is refcounted so one +device's teardown cannot clear it under a peer. + +**Status.** v2 merged through the iommu tree 2026-09-24, present in +`linux-next-20261016`, queued for **Linux 7.4** (merge window late October 2026, +stable ~end of year, likely LTS candidate). NOT available here: `/boot/ +System.map-7.0.0-38-generic` contains zero `perfopt` symbols, and Ubuntu 26.04's +archives cap at `7.0.0-38.38` for both `linux-image-generic` and +`linux-generic-hwe-26.04-edge` — reaching 7.4 means a mainline PPA or a local +build. A 2026-09-28 follow-up removes the `amdgpu.iommu_perfopt` knob and makes +PerfOpt compulsory. + +**Measured.** Phoronix (2026-09-29, Lemonade + llama.cpp, same kernel, +`amdgpu.iommu_perfopt=0` vs default): **18–23 %** on Strix Point / Krackan +laptops, **2–4 % on a Strix Halo desktop** — which is also the evidence that our +silicon implements the bit (our IOMMU is `1022:1508` Strix/Strix Halo rev 02). + +**The September number was never strict-flush cost.** Ubuntu ships +`CONFIG_IOMMU_DEFAULT_DMA_LAZY=y` (STRICT not set), so IOTLB invalidation was +already batched, and a coherent iGPU on unified memory takes no SWIOTLB bounce. +What we measured was **translated-domain** cost — and `iommu=pt` avoids that +today, with or without PerfOpt. The March attempt to make passthrough the AMD +default was rejected outright, so `iommu=pt` / `iommu.passthrough=1` stays the +sanctioned route. + +**The NPU is compatible with identity mode.** `amdxdna` binds through +`iommu_sva_bind_device()` + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain, +**independent of the default domain type** (`drivers/accel/amdxdna/aie2_pci.c`), +and PerfOpt is not on that path at all. It only needs the IOMMU *enabled* — that +is the whole `err=-19` story. Kernel pick notes: the Ubuntu 6.17 `-95` SVA +failures were a GCR3 regression fixed before our kernel, but **6.18–6.18.7 +disables SVA outright**; skip that range. + +**Container access needs no reboot to settle:** `/dev/accel/accel0` already +exists as `crw-rw---- root:render 261,0` and `dank` is in `render`, so a quadlet +wants `AddDevice=/dev/accel/accel0` plus `GroupAdd=keep-groups` — the same +pattern ComfyUI uses for `/dev/kfd`. + +**Revised procedure for the next boot-line window** (still a maintenance window, +still an explicit OK per run): + +1. `iommu=pt` in place of `amd_iommu=off` — NPU usable, translated cost gone, + residual 2–4 % on this silicon class. +2. Re-measure prefill against the **current** payload. The 13–16 % belongs to the + 2026-09-11 shape; the 2026-10-17 native-v2 boot runs 1,513–1,604 t/s cold + prefill, so the old baseline is not comparable. +3. Keep the fabric-clock pin (`pp_dpm_fclk` level 7) for GPU+NPU co-run. +4. On 7.4: nothing to do — PerfOpt arms itself when the iGPU is in the identity + domain. Confirm with `dmesg | grep -i perfopt`; force off with + `amdgpu.iommu_perfopt=0` while the knob still exists. + ## Related - [[frank-npu]] — the endpoint this retired, including the err=-19 root cause @@ -98,3 +162,9 @@ deployment — reverting one does not require reverting the other. ## Citations - [peonist-ai/halogen-flash-server](https://github.com/peonist-ai/halogen-flash-server) — README "The host settings these numbers were measured on", CHANGELOG 0.5.5–0.5.8 +- [iommu/amd: Add PerfOpt IOMMU performance optimization support (v2)](https://lists.freedesktop.org/archives/amd-gfx/2026-September/152501.html) — series 0/2 at [/153378.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153378.html), merged via the iommu tree 2026-09-24 ([/153844.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153844.html)) +- [Compulsory-PerfOpt follow-up (removes `amdgpu.iommu_perfopt`)](https://patchew.org/linux/20260928045050.955165-1-superm1@kernel.org/) +- [Phoronix: AMD Boosting AI/LLM Performance For Radeon iGPUs As Much As 18~23% With Linux 7.4](https://www.phoronix.com/review/amd-perfopt) — Strix Halo desktop arm (2–4 %), `amdgpu.iommu_perfopt=0` A/B; news write-up [here](https://www.phoronix.com/news/AMD-PerfOpt-Linux-7.4) +- [Robin Murphy rejecting the AMD passthrough-default patch](https://lkml.org/lkml/2026/3/26/924) — `CONFIG_IOMMU_DEFAULT_PASSTHROUGH` stays the sanctioned path +- [Debian wiki: XDNA](https://wiki.debian.org/XDNA) — driver + firmware upstreamed in 6.14; **amdxdna broken on 6.18–6.18.7** by an IOMMU regression that disables SVA ([report](https://lore.kernel.org/regressions/870872aa-28e9-412a-bac6-8020bf560e4f@amd.com/)) +- [amd/xdna-driver#704](https://github.com/amd/xdna-driver/issues/704) — the `err=-19` bind failure when the IOMMU is disabled