Frank
9b372acde4
Reading patch v2 1/2 verbatim: amd_iommu_enable_perfopt() rehoming needs the device ALREADY on the identity domain and switches ATS/PRI/PASID/GCR3 off via the skip_caps path. So an iGPU in identity mode still pays a 1:1 page walk plus those four capabilities until PerfOpt exists -- iommu=pt recovers most of the gap, PerfOpt the last part. The first pass claimed pt alone already avoids the cost; that was wrong and is retracted here and in the posted issue comment (edit 23). Also re-keys frank-npu.md, which carried the dead law 'iommu=pt is not a middle ground' as its retirement rationale: identity mode + SVA is exactly the revival path (amdxdna binds its own IOPT_TYPE_SVA domain), and the cost is single-digit percent on this silicon class. Procedure step 2 now demands a local off-vs-pt measurement instead of inheriting the vendor's 13-16%, whose slow-arm boot line our record never identified.
177 lines
10 KiB
Markdown
177 lines
10 KiB
Markdown
---
|
||
type: research
|
||
title: Dankclaw IOMMU translation off (amd_iommu=off) — decision record
|
||
description: Why Dankclaw boots with amd_iommu=off since 2026-09-11, what it buys (a vendor-claimed 13-16% prefill for frank-dragoon, never measured locally), what it costs (the XDNA2 NPU endpoint, machine-wide DMA translation), the 2026-10-18 AMD PerfOpt research that puts the trade on an expiring path, and how to revert.
|
||
tags:
|
||
- dankclaw
|
||
- iommu
|
||
- kernel
|
||
- grub
|
||
- prefill
|
||
- frank-dragoon
|
||
- halogen
|
||
- frank-npu
|
||
timestamp: 2026-09-11
|
||
---
|
||
|
||
# Dankclaw IOMMU translation off — decision record
|
||
|
||
**Decision (2026-09-11):** Dankclaw boots with **`amd_iommu=off`** instead of
|
||
`iommu=pt`, to recover the prefill throughput the large-model runtime loses to
|
||
IOMMU translation. The XDNA2 NPU endpoint was accepted as the cost and is
|
||
retired ([[frank-npu]]).
|
||
|
||
## What changed
|
||
|
||
`GRUB_CMDLINE_LINUX_DEFAULT` in `/etc/default/grub` on Dankclaw:
|
||
|
||
```
|
||
# before
|
||
ro quiet splash iommu=pt crashkernel=…
|
||
|
||
# after
|
||
ro quiet splash amd_iommu=off crashkernel=…
|
||
```
|
||
|
||
Then `sudo update-grub` and a reboot. Nothing else on the command line was
|
||
touched.
|
||
|
||
## Evidence
|
||
|
||
The upstream project published the measurement, and it is the only A/B they
|
||
ran: **460 → 385 tok/s prefill at a 2,048-token prompt, 13 to 16 percent**,
|
||
taken on a Ryzen AI Max+ 395 with 128 GB — the same SKU as Dankclaw. Their
|
||
published serving profile boots this way, so every number in their README is a
|
||
number with translation off. Decode is unaffected; prefill is where the loss is.
|
||
|
||
The mechanism offered is DMA translation on the path every GPU allocation
|
||
takes. The project states plainly that this is *their* configuration and not a
|
||
tuning guide, that the flag turns off DMA translation machine-wide, and that it
|
||
takes the NPU with it.
|
||
|
||
## What it costs
|
||
|
||
1. **The NPU endpoint dies.** With IOMMU off there is no PASID/SVA state and
|
||
`/dev/accel/accel0` cannot be opened — `err=-19`, the exact failure
|
||
documented in [[frank-npu]]. Audio input left the fleet with it; image input
|
||
survives on `frank-dragoon` through halogen's vision tower.
|
||
2. **Posture change machine-wide.** Dankclaw also serves Caddy (the TLS edge
|
||
and the Anubis/Authentik guard chain), Authentik, Pi-hole, Gitea, and the
|
||
observability stack. DMA translation is now off for all of them. This was
|
||
accepted explicitly as the price of the prefill figure.
|
||
|
||
## What was deliberately NOT copied
|
||
|
||
The reference machine's command line has six settings. Only one was measured,
|
||
so only that one was taken:
|
||
|
||
| setting | reference value | why not here |
|
||
|---|---|---|
|
||
| `amd_iommu=off` | set | **taken** — the measured 13–16 % |
|
||
| `amdgpu.gttsize` | `126976` MiB | a size, not a constant: it must equal installed RAM (their table: 128 GB → 126976, 96 GB → 95232, 64 GB → 63488). Dankclaw's GTT already reports **114 GiB ≈ 94 % of its 121 GiB** without the flag, so it would buy nothing |
|
||
| `ttm.pages_limit` | `32505856` | the same number in 4 KiB pages (their 128 GB row is 99.3 % of RAM). Pasting it onto a different machine asks the driver for more GTT than exists |
|
||
| `amdgpu.vm_update_mode=0` | set | upstream has never run without it, makes no claim about what it buys, and lists it "for completeness rather than recommended" |
|
||
| `amdgpu.noretry=0` | set | as above |
|
||
| `amdgpu.sg_display=0` | set | as above |
|
||
|
||
## Verification and revert
|
||
|
||
```bash
|
||
cat /proc/cmdline # expect amd_iommu=off, no iommu=pt
|
||
ls /sys/kernel/iommu_groups/ | wc -l # expect 0 groups (was 30)
|
||
systemctl is-active frank-npu # expect inactive (masked)
|
||
curl -s http://127.0.0.1:7770/health # frank-dragoon must still answer 200
|
||
```
|
||
|
||
**To revert:** put `iommu=pt` back in `GRUB_CMDLINE_LINUX_DEFAULT` in place of
|
||
`amd_iommu=off`, run `sudo update-grub`, reboot, and unmask/start `frank-npu`
|
||
with the archived unit (`.AGENTTRASH/frank-npu.service.flm-npu`) if the NPU
|
||
endpoint is wanted back. The IOMMU change is independent of the halogen
|
||
deployment — reverting one does not require reverting the other.
|
||
|
||
## 2026-10-18 research: the penalty now has a hardware fix (AMD PerfOpt)
|
||
|
||
Asked and answered — **the decision changes, the host does not.** Nothing was
|
||
changed on Dankclaw today; this section records why the figure behind the
|
||
decision is on an expiring path — and that it was never measured on this host.
|
||
|
||
**Mechanism.** AMD IOMMU **PerfOpt**, spec-defined in the AMD I/O Virtualization
|
||
Technology (IOMMU) spec §3.4.9: MMIO `0x016C` bit 13 `PERF_OPT_EN`, gated by
|
||
extended-feature bit 45 `FEATURE_PERF_OPT`. Armed, it stops GPA→SPA translation
|
||
for the *privileged integrated GPU* — only the IR/IW permission bits are
|
||
enforced — while the IOMMU stays fully enabled for every other device, the NPU
|
||
included. Driver: `drivers/iommu/amd/{iommu.c,init.c,amd_iommu_types.h}`. The only
|
||
consumer is **amdgpu**: `amdgpu_device_use_perfopt()` requires `AMD_IS_APU` **and
|
||
the identity domain**, and the shared per-IOMMU bit is refcounted so one
|
||
device's teardown cannot clear it under a peer.
|
||
|
||
**Status.** v2 merged through the iommu tree 2026-09-24, present in
|
||
`linux-next-20261016`, queued for **Linux 7.4** (merge window late October 2026,
|
||
stable ~end of year, likely LTS candidate). NOT available here: `/boot/
|
||
System.map-7.0.0-38-generic` contains zero `perfopt` symbols, and Ubuntu 26.04's
|
||
archives cap at `7.0.0-38.38` for both `linux-image-generic` and
|
||
`linux-generic-hwe-26.04-edge` — reaching 7.4 means a mainline PPA or a local
|
||
build. A 2026-09-28 follow-up removes the `amdgpu.iommu_perfopt` knob and makes
|
||
PerfOpt compulsory.
|
||
|
||
**Measured.** Phoronix (2026-09-29, Lemonade + llama.cpp, same kernel,
|
||
`amdgpu.iommu_perfopt=0` vs default): **18–23 %** on Strix Point / Krackan
|
||
laptops, **2–4 % on a Strix Halo desktop** — which is also the evidence that our
|
||
silicon implements the bit (our IOMMU is `1022:1508` Strix/Strix Halo rev 02).
|
||
|
||
**Identity mode is not "no IOMMU".** Ubuntu ships `CONFIG_IOMMU_DEFAULT_DMA_LAZY=y`
|
||
(STRICT not set), so IOTLB invalidation was already batched, and a coherent iGPU
|
||
on unified memory takes no SWIOTLB bounce — strict flush and bounce buffering do
|
||
not explain the September figure. But a device attached to the identity domain
|
||
still has its DMA walked through 1:1 page tables, and the AMD attach path enables
|
||
**ATS/PRI/PASID/GCR3** for it; PerfOpt's own enable path has to switch those four
|
||
off again (the `skip_caps` label in `attach_device`) to get the speedup, and the
|
||
cover letter says it arms only "when an integrated GPU is **already in identity
|
||
mode**". So `pt` is most of the recovery and PerfOpt is the last part of it.
|
||
|
||
**The NPU is compatible with identity mode.** `amdxdna` binds through
|
||
`iommu_sva_bind_device()` + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain,
|
||
**independent of the default domain type** (`drivers/accel/amdxdna/aie2_pci.c`),
|
||
and PerfOpt is not on that path at all. It only needs the IOMMU *enabled* — that
|
||
is the whole `err=-19` story. Kernel pick notes: the Ubuntu 6.17 `-95` SVA
|
||
failures were a GCR3 regression fixed before our kernel, but **6.18–6.18.7
|
||
disables SVA outright**; skip that range.
|
||
|
||
**Container access needs no reboot to settle:** `/dev/accel/accel0` already
|
||
exists as `crw-rw---- root:render 261,0` and `dank` is in `render`, so a quadlet
|
||
wants `AddDevice=/dev/accel/accel0` plus `GroupAdd=keep-groups` — the same
|
||
pattern ComfyUI uses for `/dev/kfd`.
|
||
|
||
**Revised procedure for the next boot-line window** (still a maintenance window,
|
||
still an explicit OK per run):
|
||
|
||
1. `iommu=pt` in place of `amd_iommu=off`. The NPU becomes usable at that reboot;
|
||
the iGPU pays the identity-walk cost until PerfOpt exists on this kernel, which
|
||
the desktop-class measurement prices in single-digit percent.
|
||
2. **Measure `off` vs `pt` on this host, on the current payload** (bench law: stop
|
||
+ mask `frank-dragoon` before any GPU A/B). The 13–16 % was the vendor's A/B
|
||
against *their* previous boot line, which our record never identified — the
|
||
2026-09-11 adoption copied their figure without a local run. The current
|
||
native-v2 payload does 1,513–1,604 t/s cold prefill, so that baseline is not
|
||
comparable regardless.
|
||
3. Keep the fabric-clock pin (`pp_dpm_fclk` level 7) for GPU+NPU co-run.
|
||
4. On 7.4: nothing to do — PerfOpt arms itself when the iGPU is in the identity
|
||
domain. Confirm with `dmesg | grep -i perfopt`; force off with
|
||
`amdgpu.iommu_perfopt=0` while the knob still exists.
|
||
|
||
## Related
|
||
|
||
- [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
|
||
- [[halogen-flash-server]] — the runtime bought by the prefill figure
|
||
- [[kingjones-strix-rocmfp4]] — the runtime it replaced (rollback target)
|
||
|
||
## Citations
|
||
|
||
- [peonist-ai/halogen-flash-server](https://github.com/peonist-ai/halogen-flash-server) — README "The host settings these numbers were measured on", CHANGELOG 0.5.5–0.5.8
|
||
- [iommu/amd: Add PerfOpt IOMMU performance optimization support (v2)](https://lists.freedesktop.org/archives/amd-gfx/2026-September/152501.html) — series 0/2 at [/153378.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153378.html), merged via the iommu tree 2026-09-24 ([/153844.html](https://lists.freedesktop.org/archives/amd-gfx/2026-September/153844.html))
|
||
- [Compulsory-PerfOpt follow-up (removes `amdgpu.iommu_perfopt`)](https://patchew.org/linux/20260928045050.955165-1-superm1@kernel.org/)
|
||
- [Phoronix: AMD Boosting AI/LLM Performance For Radeon iGPUs As Much As 18~23% With Linux 7.4](https://www.phoronix.com/review/amd-perfopt) — Strix Halo desktop arm (2–4 %), `amdgpu.iommu_perfopt=0` A/B; news write-up [here](https://www.phoronix.com/news/AMD-PerfOpt-Linux-7.4)
|
||
- [Robin Murphy rejecting the AMD passthrough-default patch](https://lkml.org/lkml/2026/3/26/924) — `CONFIG_IOMMU_DEFAULT_PASSTHROUGH` stays the sanctioned path
|
||
- [Debian wiki: XDNA](https://wiki.debian.org/XDNA) — driver + firmware upstreamed in 6.14; **amdxdna broken on 6.18–6.18.7** by an IOMMU regression that disables SVA ([report](https://lore.kernel.org/regressions/870872aa-28e9-412a-bac6-8020bf560e4f@amd.com/))
|
||
- [amd/xdna-driver#704](https://github.com/amd/xdna-driver/issues/704) — the `err=-19` bind failure when the IOMMU is disabled
|