docs(iommu): correct the identity-mode cost claim; pt is most of the fix, PerfOpt the last part

Reading patch v2 1/2 verbatim: amd_iommu_enable_perfopt() rehoming needs the
device ALREADY on the identity domain and switches ATS/PRI/PASID/GCR3 off via
the skip_caps path. So an iGPU in identity mode still pays a 1:1 page walk plus
those four capabilities until PerfOpt exists -- iommu=pt recovers most of the
gap, PerfOpt the last part. The first pass claimed pt alone already avoids the
cost; that was wrong and is retracted here and in the posted issue comment
(edit 23).

Also re-keys frank-npu.md, which carried the dead law 'iommu=pt is not a middle
ground' as its retirement rationale: identity mode + SVA is exactly the revival
path (amdxdna binds its own IOPT_TYPE_SVA domain), and the cost is single-digit
percent on this silicon class. Procedure step 2 now demands a local off-vs-pt
measurement instead of inheriting the vendor's 13-16%, whose slow-arm boot line
our record never identified.
This commit is contained in:
Frank
2026-10-18 01:03:19 -07:00
parent 7643381b9b
commit 9b372acde4
4 changed files with 106 additions and 28 deletions
+59
View File
@@ -2007,3 +2007,62 @@ every restart since the cutover.
200,704-position pool behind both slots; two of ≤100k still do). Bought: 1.7
GiB back as page cache for the 47.68 GiB n-gram table — 9.1 GiB against the
2026-10-06 shape's 524,288 pool — and a smaller prompt-cache entry.
## 2026-10-18 — issue #3 research: AMD PerfOpt retires the reason for `amd_iommu=off`
- Ask: does a recent kernel change remove the IOMMU translation penalty that
justifies the 2026-09-11 `amd_iommu=off` boot line, i.e. can the XDNA2 NPU come
back? Answered and documented; **no host change** — the fix is not installable
from the Ubuntu 26.04 archive.
- The change is **AMD PerfOpt** (spec §3.4.9, MMIO `0x016C` bit 13, gated by
extended-feature bit 45): merged via the iommu tree 2026-09-24, first release
**Linux 7.4**. It stops GPA→SPA translation for the *privileged iGPU* only and
leaves the IOMMU fully armed for every other device, NPU included. The only
consumer is `amdgpu_device_use_perfopt()`, which requires `AMD_IS_APU` **and the
identity domain** — so PerfOpt rides on top of `iommu=pt`, it does not replace
it. A 2026-09-28 follow-up deletes the `amdgpu.iommu_perfopt` knob and makes it
compulsory.
- Local census (read-only): kernel `7.0.0-38-generic`; **zero** `perfopt` symbols
in its System.map; `linux-image-generic` and `linux-generic-hwe-26.04-edge` both
pin `7.0.0-38.38` → 7.4 needs a mainline PPA or a local build. `CONFIG_IOMMU_
DEFAULT_DMA_LAZY=y` (STRICT unset). `/dev/accel/accel0` already exists as
`crw-rw---- root:render 261,0` with `dank` in `render`, so the quadlet law is
`AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups` — the ComfyUI `/dev/kfd`
pattern, no new udev rule.
- **Three premises of the issue corrected.** (1) The 13–16 % is the *vendor's* A/B
against *their* previous boot line, which our record never identified — we
copied the figure on 2026-09-11 without ever measuring here. Strict flush and
bounce buffering are not the mechanism either (Ubuntu ships lazy DMA, and a
coherent iGPU on unified memory takes no SWIOTLB bounce). What identity mode
actually costs the iGPU is the 1:1 page walk plus **ATS/PRI/PASID/GCR3** being
switched on at attach — the same four the PerfOpt enable path switches back off
(the `skip_caps` label in `attach_device`), and why it arms only "when an
integrated GPU is already in identity mode". (2) The NPU does **not** have to
wait for 7.4: `iommu=pt` enables the IOMMU, which is all `err=-19` asks for.
Making passthrough the AMD default was rejected on the list (Robin Murphy,
2026-03-26), so `pt` stays the sanctioned route. (3) The 460 tok/s baseline is
stale — the 2026-10-17 native-v2 payload runs **1,513–1,604 t/s cold prefill** —
so the window measures `off` vs `pt` locally and inherits no number.
- Silicon check: our IOMMU is `1022:1508` Strix/Strix Halo rev 02, and Phoronix's
**Strix Halo desktop** arm measured only **2–4 %** from PerfOpt (18–23 % on
Strix Point / Krackan laptops) — the modest arm is the one that describes this
box. Kernel pick: **skip 6.18–6.18.7**, where an IOMMU regression disables SVA
outright.
- NPU legality under identity mode: `amdxdna` binds via `iommu_sva_bind_device()`
+ PASID + GCR3 into its own `IOPT_TYPE_SVA` domain, independent of the default
domain type (`drivers/accel/amdxdna/aie2_pci.c`); `err=-19` needs only an
*enabled* IOMMU, which `pt` gives.
- Self-correction inside the same session: the first pass — and the first version
of the posted comment — claimed `iommu=pt` alone already avoids the cost. It
does not. Patch v2 1/2 shows an identity attach keeps the page walk and enables
ATS/PRI/PASID/GCR3, so `pt` recovers **most** of the gap and PerfOpt is the last
part of it. The comment was edited to the corrected text.
- Artifacts: new section in
`serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` (+6 citations,
front-matter re-keyed to "vendor-claimed, never measured locally"),
`frank-npu.md` § *Why it is retired* corrected — the "`pt` is not a middle
ground" law is dead — the AGENTS.md key-lesson bullet re-keyed, a 4-step
maintenance-window procedure that measures locally instead of inheriting the
vendor number, and the plan posted to Gitea issue #3 (comment 23, then edited).
bats 371/371; commits `7643381` plus this correction. Nothing deployed — no
service was touched.