Frank
7643381b9b
Records the 2026-10-18 answer to 'does a recent kernel change retire amd_iommu=off': PerfOpt (iommu tree 2026-09-24, Linux 7.4) stops GPA->SPA translation for the privileged iGPU while the IOMMU stays enabled for every other device, the NPU included. Local census proves the fix is absent here (zero perfopt symbols in System.map-7.0.0-38, archive caps at 7.0.0-38.38), and the September 13-16% figure was translated-domain cost, not strict-flush cost -- lazy DMA is the Ubuntu default, so iommu=pt already avoids it. The corrected procedure therefore flips to iommu=pt now and treats PerfOpt as a 2-4% later bonus on this silicon class (Phoronix desktop arm), with the NPU legal under identity mode because amdxdna binds its own SVA domain. Re-keys the AGENTS.md key-lesson bullet; no host change.
171 lines
9.4 KiB
Markdown
171 lines
9.4 KiB
Markdown
---
|
||
type: research
|
||
title: Dankclaw IOMMU translation off (amd_iommu=off) — decision record
|
||
description: Why Dankclaw boots with amd_iommu=off since 2026-09-11, what it buys (13-16% prefill for frank-dragoon), what it costs (the XDNA2 NPU endpoint, machine-wide DMA translation), what was deliberately not copied from the reference machine, and how to revert.
|
||
tags:
|
||
- dankclaw
|
||
- iommu
|
||
- kernel
|
||
- grub
|
||
- prefill
|
||
- frank-dragoon
|
||
- halogen
|
||
- frank-npu
|
||
timestamp: 2026-09-11
|
||
---
|
||
|
||
# Dankclaw IOMMU translation off — decision record
|
||
|
||
**Decision (2026-09-11):** Dankclaw boots with **`amd_iommu=off`** instead of
|
||
`iommu=pt`, to recover the prefill throughput the large-model runtime loses to
|
||
IOMMU translation. The XDNA2 NPU endpoint was accepted as the cost and is
|
||
retired ([[frank-npu]]).
|
||
|
||
## What changed
|
||
|
||
`GRUB_CMDLINE_LINUX_DEFAULT` in `/etc/default/grub` on Dankclaw:
|
||
|
||
```
|
||
# before
|
||
ro quiet splash iommu=pt crashkernel=…
|
||
|
||
# after
|
||
ro quiet splash amd_iommu=off crashkernel=…
|
||
```
|
||
|
||
Then `sudo update-grub` and a reboot. Nothing else on the command line was
|
||
touched.
|
||
|
||
## Evidence
|
||
|
||
The upstream project published the measurement, and it is the only A/B they
|
||
ran: **460 → 385 tok/s prefill at a 2,048-token prompt, 13 to 16 percent**,
|
||
taken on a Ryzen AI Max+ 395 with 128 GB — the same SKU as Dankclaw. Their
|
||
published serving profile boots this way, so every number in their README is a
|
||
number with translation off. Decode is unaffected; prefill is where the loss is.
|
||
|
||
The mechanism offered is DMA translation on the path every GPU allocation
|
||
takes. The project states plainly that this is *their* configuration and not a
|
||
tuning guide, that the flag turns off DMA translation machine-wide, and that it
|
||
takes the NPU with it.
|
||
|
||
## What it costs
|
||
|
||
1. **The NPU endpoint dies.** With IOMMU off there is no PASID/SVA state and
|
||
`/dev/accel/accel0` cannot be opened — `err=-19`, the exact failure
|
||
documented in [[frank-npu]]. Audio input left the fleet with it; image input
|
||
survives on `frank-dragoon` through halogen's vision tower.
|
||
2. **Posture change machine-wide.** Dankclaw also serves Caddy (the TLS edge
|
||
and the Anubis/Authentik guard chain), Authentik, Pi-hole, Gitea, and the
|
||
observability stack. DMA translation is now off for all of them. This was
|
||
accepted explicitly as the price of the prefill figure.
|
||
|
||
## What was deliberately NOT copied
|
||
|
||
The reference machine's command line has six settings. Only one was measured,
|
||
so only that one was taken:
|
||
|
||
| setting | reference value | why not here |
|
||
|---|---|---|
|
||
| `amd_iommu=off` | set | **taken** — the measured 13–16 % |
|
||
| `amdgpu.gttsize` | `126976` MiB | a size, not a constant: it must equal installed RAM (their table: 128 GB → 126976, 96 GB → 95232, 64 GB → 63488). Dankclaw's GTT already reports **114 GiB ≈ 94 % of its 121 GiB** without the flag, so it would buy nothing |
|
||
| `ttm.pages_limit` | `32505856` | the same number in 4 KiB pages (their 128 GB row is 99.3 % of RAM). Pasting it onto a different machine asks the driver for more GTT than exists |
|
||
| `amdgpu.vm_update_mode=0` | set | upstream has never run without it, makes no claim about what it buys, and lists it "for completeness rather than recommended" |
|
||
| `amdgpu.noretry=0` | set | as above |
|
||
| `amdgpu.sg_display=0` | set | as above |
|
||
|
||
## Verification and revert
|
||
|
||
```bash
|
||
cat /proc/cmdline # expect amd_iommu=off, no iommu=pt
|
||
ls /sys/kernel/iommu_groups/ | wc -l # expect 0 groups (was 30)
|
||
systemctl is-active frank-npu # expect inactive (masked)
|
||
curl -s http://127.0.0.1:7770/health # frank-dragoon must still answer 200
|
||
```
|
||
|
||
**To revert:** put `iommu=pt` back in `GRUB_CMDLINE_LINUX_DEFAULT` in place of
|
||
`amd_iommu=off`, run `sudo update-grub`, reboot, and unmask/start `frank-npu`
|
||
with the archived unit (`.AGENTTRASH/frank-npu.service.flm-npu`) if the NPU
|
||
endpoint is wanted back. The IOMMU change is independent of the halogen
|
||
deployment — reverting one does not require reverting the other.
|
||
|
||
## 2026-10-18 research: the penalty now has a hardware fix (AMD PerfOpt)
|
||
|
||
Asked and answered — **the decision changes, the host does not.** Nothing was
|
||
changed on Dankclaw today; this section records why the 13–16 % figure is on an
|
||
expiring path.
|
||
|
||
**Mechanism.** AMD IOMMU **PerfOpt**, spec-defined in the AMD I/O Virtualization
|
||
Technology (IOMMU) spec §3.4.9: MMIO `0x016C` bit 13 `PERF_OPT_EN`, gated by
|
||
extended-feature bit 45 `FEATURE_PERF_OPT`. Armed, it stops GPA→SPA translation
|
||
for the *privileged integrated GPU* — only the IR/IW permission bits are
|
||
enforced — while the IOMMU stays fully enabled for every other device, the NPU
|
||
included. Driver: `drivers/iommu/amd/{iommu.c,amd_iommu_types.h}`. The only
|
||
consumer is **amdgpu**: `amdgpu_device_use_perfopt()` requires `AMD_IS_APU` **and
|
||
the identity domain**, and the shared per-IOMMU bit is refcounted so one
|
||
device's teardown cannot clear it under a peer.
|
||
|
||
**Status.** v2 merged through the iommu tree 2026-09-24, present in
|
||
`linux-next-20261016`, queued for **Linux 7.4** (merge window late October 2026,
|
||
stable ~end of year, likely LTS candidate). NOT available here: `/boot/
|
||
System.map-7.0.0-38-generic` contains zero `perfopt` symbols, and Ubuntu 26.04's
|
||
archives cap at `7.0.0-38.38` for both `linux-image-generic` and
|
||
`linux-generic-hwe-26.04-edge` — reaching 7.4 means a mainline PPA or a local
|
||
build. A 2026-09-28 follow-up removes the `amdgpu.iommu_perfopt` knob and makes
|
||
PerfOpt compulsory.
|
||
|
||
**Measured.** Phoronix (2026-09-29, Lemonade + llama.cpp, same kernel,
|
||
`amdgpu.iommu_perfopt=0` vs default): **18–23 %** on Strix Point / Krackan
|
||
laptops, **2–4 % on a Strix Halo desktop** — which is also the evidence that our
|
||
silicon implements the bit (our IOMMU is `1022:1508` Strix/Strix Halo rev 02).
|
||
|
||
**The September number was never strict-flush cost.** Ubuntu ships
|
||
`CONFIG_IOMMU_DEFAULT_DMA_LAZY=y` (STRICT not set), so IOTLB invalidation was
|
||
already batched, and a coherent iGPU on unified memory takes no SWIOTLB bounce.
|
||
What we measured was **translated-domain** cost — and `iommu=pt` avoids that
|
||
today, with or without PerfOpt. The March attempt to make passthrough the AMD
|
||
default was rejected outright, so `iommu=pt` / `iommu.passthrough=1` stays the
|
||
sanctioned route.
|
||
|
||
**The NPU is compatible with identity mode.** `amdxdna` binds through
|
||
`iommu_sva_bind_device()` + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain,
|
||
**independent of the default domain type** (`drivers/accel/amdxdna/aie2_pci.c`),
|
||
and PerfOpt is not on that path at all. It only needs the IOMMU *enabled* — that
|
||
is the whole `err=-19` story. Kernel pick notes: the Ubuntu 6.17 `-95` SVA
|
||
failures were a GCR3 regression fixed before our kernel, but **6.18–6.18.7
|
||
disables SVA outright**; skip that range.
|
||
|
||
**Container access needs no reboot to settle:** `/dev/accel/accel0` already
|
||
exists as `crw-rw---- root:render 261,0` and `dank` is in `render`, so a quadlet
|
||
wants `AddDevice=/dev/accel/accel0` plus `GroupAdd=keep-groups` — the same
|
||
pattern ComfyUI uses for `/dev/kfd`.
|
||
|
||
**Revised procedure for the next boot-line window** (still a maintenance window,
|
||
still an explicit OK per run):
|
||
|
||
1. `iommu=pt` in place of `amd_iommu=off` — NPU usable, translated cost gone,
|
||
residual 2–4 % on this silicon class.
|
||
2. Re-measure prefill against the **current** payload. The 13–16 % belongs to the
|
||
2026-09-11 shape; the 2026-10-17 native-v2 boot runs 1,513–1,604 t/s cold
|
||
prefill, so the old baseline is not comparable.
|
||
3. Keep the fabric-clock pin (`pp_dpm_fclk` level 7) for GPU+NPU co-run.
|
||
4. On 7.4: nothing to do — PerfOpt arms itself when the iGPU is in the identity
|
||
domain. Confirm with `dmesg | grep -i perfopt`; force off with
|
||
`amdgpu.iommu_perfopt=0` while the knob still exists.
|
||
|
||
## Related
|
||
|
||
- [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
|
||
- [[halogen-flash-server]] — the runtime bought by the prefill figure
|
||
- [[kingjones-strix-rocmfp4]] — the runtime it replaced (rollback target)
|
||
|
||
## Citations
|
||
|
||
- [peonist-ai/halogen-flash-server](https://github.com/peonist-ai/halogen-flash-server) — README "The host settings these numbers were measured on", CHANGELOG 0.5.5–0.5.8
|
||
- [iommu/amd: Add PerfOpt IOMMU performance optimization support (v2)](https://lists.freedesktop.org/archives/amd-gfx/2026-September/152501.html) — series 0/2 at [/153378.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153378.html), merged via the iommu tree 2026-09-24 ([/153844.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153844.html))
|
||
- [Compulsory-PerfOpt follow-up (removes `amdgpu.iommu_perfopt`)](https://patchew.org/linux/20260928045050.955165-1-superm1@kernel.org/)
|
||
- [Phoronix: AMD Boosting AI/LLM Performance For Radeon iGPUs As Much As 18~23% With Linux 7.4](https://www.phoronix.com/review/amd-perfopt) — Strix Halo desktop arm (2–4 %), `amdgpu.iommu_perfopt=0` A/B; news write-up [here](https://www.phoronix.com/news/AMD-PerfOpt-Linux-7.4)
|
||
- [Robin Murphy rejecting the AMD passthrough-default patch](https://lkml.org/lkml/2026/3/26/924) — `CONFIG_IOMMU_DEFAULT_PASSTHROUGH` stays the sanctioned path
|
||
- [Debian wiki: XDNA](https://wiki.debian.org/XDNA) — driver + firmware upstreamed in 6.14; **amdxdna broken on 6.18–6.18.7** by an IOMMU regression that disables SVA ([report](https://lore.kernel.org/regressions/870872aa-28e9-412a-bac6-8020bf560e4f@amd.com/))
|
||
- [amd/xdna-driver#704](https://github.com/amd/xdna-driver/issues/704) — the `err=-19` bind failure when the IOMMU is disabled
|