docs(iommu): correct the identity-mode cost claim; pt is most of the fix, PerfOpt the last part

Reading patch v2 1/2 verbatim: amd_iommu_enable_perfopt() rehoming needs the
device ALREADY on the identity domain and switches ATS/PRI/PASID/GCR3 off via
the skip_caps path. So an iGPU in identity mode still pays a 1:1 page walk plus
those four capabilities until PerfOpt exists -- iommu=pt recovers most of the
gap, PerfOpt the last part. The first pass claimed pt alone already avoids the
cost; that was wrong and is retracted here and in the posted issue comment
(edit 23).

Also re-keys frank-npu.md, which carried the dead law 'iommu=pt is not a middle
ground' as its retirement rationale: identity mode + SVA is exactly the revival
path (amdxdna binds its own IOPT_TYPE_SVA domain), and the cost is single-digit
percent on this silicon class. Procedure step 2 now demands a local off-vs-pt
measurement instead of inheriting the vendor's 13-16%, whose slow-arm boot line
our record never identified.

AGENTS.md

@@ -1107,15 +1107,16 @@ linked runbooks.
   times (`RuntimeError: No HIP GPUs are available`).
 - SearXNG outbound traffic must stay on the Gluetun tunnel; do not move its
   ports off the gluetun service.
-- `amd_iommu=off` on Dankclaw (bought 13–16 % prefill for frank-dragoon,
+- `amd_iommu=off` on Dankclaw (adopted 2026-09-11 on the vendor's 13–16 %
-  2026-09-11) retires the XDNA2 NPU: the driver's SVA/PASID bind needs an
+  prefill figure, never measured here) retires the XDNA2 NPU: the driver's
-  enabled IOMMU (`/dev/accel/accel0` fails `err=-19`). That trade is on an
+  SVA/PASID bind needs an enabled IOMMU (`/dev/accel/accel0` fails `err=-19`).
-  expiring path: **AMD PerfOpt** (iommu tree 2026-09-24, Linux 7.4) stops
+  That trade is on an expiring path: **AMD PerfOpt** (iommu tree 2026-09-24,
-  iGPU GPA→SPA translation while the IOMMU stays enabled for every other
+  Linux 7.4) makes the IOMMU skip GPA→SPA translation for the *privileged iGPU*
-  device, so the next boot-line window is `iommu=pt` with the NPU usable —
+  while staying fully armed for every other device — and it arms only when the
-  measured only 2–4 % residual on a Strix Halo desktop (18–23 % on Strix
+  iGPU is already in the identity domain, so the next boot-line window is
-  Point laptops). Not installable here yet: Ubuntu 26.04 caps at
+  `iommu=pt` with the NPU usable, not `off`. PerfOpt buys 2–4 % back on a Strix
-  `7.0.0-38`. Full research in
+  Halo desktop (18–23 % on Strix Point laptops); it is not installable here
+  yet (Ubuntu 26.04 caps at `7.0.0-38`). Research in
   `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md`.
 - Rootless Quadlet containers run with `--replace --rm`: `podman logs` covers
   only the CURRENT instance; history across recreations lives in

serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md

@@ -1,7 +1,7 @@
 ---
 type: research
 title: Dankclaw IOMMU translation off (amd_iommu=off) — decision record
-description: Why Dankclaw boots with amd_iommu=off since 2026-09-11, what it buys (13-16% prefill for frank-dragoon), what it costs (the XDNA2 NPU endpoint, machine-wide DMA translation), what was deliberately not copied from the reference machine, and how to revert.
+description: Why Dankclaw boots with amd_iommu=off since 2026-09-11, what it buys (a vendor-claimed 13-16% prefill for frank-dragoon, never measured locally), what it costs (the XDNA2 NPU endpoint, machine-wide DMA translation), the 2026-10-18 AMD PerfOpt research that puts the trade on an expiring path, and how to revert.
 tags:
   - dankclaw
   - iommu
@@ -92,15 +92,15 @@ deployment — reverting one does not require reverting the other.
 ## 2026-10-18 research: the penalty now has a hardware fix (AMD PerfOpt)
 
 Asked and answered — **the decision changes, the host does not.** Nothing was
-changed on Dankclaw today; this section records why the 13–16 % figure is on an
+changed on Dankclaw today; this section records why the figure behind the
-expiring path.
+decision is on an expiring path — and that it was never measured on this host.
 
 **Mechanism.** AMD IOMMU **PerfOpt**, spec-defined in the AMD I/O Virtualization
 Technology (IOMMU) spec §3.4.9: MMIO `0x016C` bit 13 `PERF_OPT_EN`, gated by
 extended-feature bit 45 `FEATURE_PERF_OPT`. Armed, it stops GPA→SPA translation
 for the *privileged integrated GPU* — only the IR/IW permission bits are
 enforced — while the IOMMU stays fully enabled for every other device, the NPU
-included. Driver: `drivers/iommu/amd/{iommu.c,amd_iommu_types.h}`. The only
+included. Driver: `drivers/iommu/amd/{iommu.c,init.c,amd_iommu_types.h}`. The only
 consumer is **amdgpu**: `amdgpu_device_use_perfopt()` requires `AMD_IS_APU` **and
 the identity domain**, and the shared per-IOMMU bit is refcounted so one
 device's teardown cannot clear it under a peer.
@@ -119,13 +119,15 @@ PerfOpt compulsory.
 laptops, **2–4 % on a Strix Halo desktop** — which is also the evidence that our
 silicon implements the bit (our IOMMU is `1022:1508` Strix/Strix Halo rev 02).
 
-**The September number was never strict-flush cost.** Ubuntu ships
+**Identity mode is not "no IOMMU".** Ubuntu ships `CONFIG_IOMMU_DEFAULT_DMA_LAZY=y`
-`CONFIG_IOMMU_DEFAULT_DMA_LAZY=y` (STRICT not set), so IOTLB invalidation was
+(STRICT not set), so IOTLB invalidation was already batched, and a coherent iGPU
-already batched, and a coherent iGPU on unified memory takes no SWIOTLB bounce.
+on unified memory takes no SWIOTLB bounce — strict flush and bounce buffering do
-What we measured was **translated-domain** cost — and `iommu=pt` avoids that
+not explain the September figure. But a device attached to the identity domain
-today, with or without PerfOpt. The March attempt to make passthrough the AMD
+still has its DMA walked through 1:1 page tables, and the AMD attach path enables
-default was rejected outright, so `iommu=pt` / `iommu.passthrough=1` stays the
+**ATS/PRI/PASID/GCR3** for it; PerfOpt's own enable path has to switch those four
-sanctioned route.
+off again (the `skip_caps` label in `attach_device`) to get the speedup, and the
+cover letter says it arms only "when an integrated GPU is **already in identity
+mode**". So `pt` is most of the recovery and PerfOpt is the last part of it.
 
 **The NPU is compatible with identity mode.** `amdxdna` binds through
 `iommu_sva_bind_device()` + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain,
@@ -143,11 +145,15 @@ pattern ComfyUI uses for `/dev/kfd`.
 **Revised procedure for the next boot-line window** (still a maintenance window,
 still an explicit OK per run):
 
-1. `iommu=pt` in place of `amd_iommu=off` — NPU usable, translated cost gone,
+1. `iommu=pt` in place of `amd_iommu=off`. The NPU becomes usable at that reboot;
-   residual 2–4 % on this silicon class.
+   the iGPU pays the identity-walk cost until PerfOpt exists on this kernel, which
-2. Re-measure prefill against the **current** payload. The 13–16 % belongs to the
+   the desktop-class measurement prices in single-digit percent.
-   2026-09-11 shape; the 2026-10-17 native-v2 boot runs 1,513–1,604 t/s cold
+2. **Measure `off` vs `pt` on this host, on the current payload** (bench law: stop
-   prefill, so the old baseline is not comparable.
+   + mask `frank-dragoon` before any GPU A/B). The 13–16 % was the vendor's A/B
+   against *their* previous boot line, which our record never identified — the
+   2026-09-11 adoption copied their figure without a local run. The current
+   native-v2 payload does 1,513–1,604 t/s cold prefill, so that baseline is not
+   comparable regardless.
 3. Keep the fabric-clock pin (`pp_dpm_fclk` level 7) for GPU+NPU co-run.
 4. On 7.4: nothing to do — PerfOpt arms itself when the iGPU is in the identity
    domain. Confirm with `dmesg | grep -i perfopt`; force off with

serverAppConfigs-docs/LLM/frank-npu.md

@@ -42,9 +42,21 @@ upstream project on this exact silicon. That single flag is the whole decision:
   out `/dev/accel/accel0` — every open fails with `err=-19` ("No such device").
   That is precisely the failure the endpoint was rescued from on 2026-08-30,
   recorded in the next section.
-- `iommu=pt` is not a middle ground for the NPU *and* the prefill number: the
+- `iommu=pt` was recorded here as "not a middle ground", on the theory that the
-  passthrough domain still costs the prefill throughput the vendor measured.
+  passthrough domain still costs the vendor's prefill number. **That claim did
-- The operator chose prefill, so the NPU endpoint goes.
+  not survive the 2026-10-18 research.** Identity mode is not free for the iGPU
+  — its DMA still walks 1:1 page tables and the AMD attach path enables
+  ATS/PRI/PASID/GCR3 for the device — but on this silicon class the cost is
+  single-digit percent, and **AMD PerfOpt** (Linux 7.4, merged 2026-09-24)
+  removes it: the IOMMU then enforces only the IR/IW permission bits for the
+  privileged iGPU and stays fully armed for every other device, the NPU
+  included. So `pt` IS the revival path for this endpoint. See
+  [[dankclaw-iommu-translation-off]] § *2026-10-18 research* for the kernel
+  census, the versions to avoid (6.18–6.18.7 disable SVA), the device-access
+  quadlet law (`AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups`), and the
+  measurement step that must precede any decision.
+- The operator took the vendor's prefill figure on the September evidence, so the
+  NPU endpoint went.
 
 Consequences accepted at retirement: **audio input left the fleet with no
 replacement** (halogen reads images only, no audio or video), and the surviving

serverAppConfigs-docs/log.md

@@ -2007,3 +2007,62 @@ every restart since the cutover.
   200,704-position pool behind both slots; two of ≤100k still do). Bought: 1.7
   GiB back as page cache for the 47.68 GiB n-gram table — 9.1 GiB against the
   2026-10-06 shape's 524,288 pool — and a smaller prompt-cache entry.
+
+## 2026-10-18 — issue #3 research: AMD PerfOpt retires the reason for `amd_iommu=off`
+
+- Ask: does a recent kernel change remove the IOMMU translation penalty that
+  justifies the 2026-09-11 `amd_iommu=off` boot line, i.e. can the XDNA2 NPU come
+  back? Answered and documented; **no host change** — the fix is not installable
+  from the Ubuntu 26.04 archive.
+- The change is **AMD PerfOpt** (spec §3.4.9, MMIO `0x016C` bit 13, gated by
+  extended-feature bit 45): merged via the iommu tree 2026-09-24, first release
+  **Linux 7.4**. It stops GPA→SPA translation for the *privileged iGPU* only and
+  leaves the IOMMU fully armed for every other device, NPU included. The only
+  consumer is `amdgpu_device_use_perfopt()`, which requires `AMD_IS_APU` **and the
+  identity domain** — so PerfOpt rides on top of `iommu=pt`, it does not replace
+  it. A 2026-09-28 follow-up deletes the `amdgpu.iommu_perfopt` knob and makes it
+  compulsory.
+- Local census (read-only): kernel `7.0.0-38-generic`; **zero** `perfopt` symbols
+  in its System.map; `linux-image-generic` and `linux-generic-hwe-26.04-edge` both
+  pin `7.0.0-38.38` → 7.4 needs a mainline PPA or a local build. `CONFIG_IOMMU_
+  DEFAULT_DMA_LAZY=y` (STRICT unset). `/dev/accel/accel0` already exists as
+  `crw-rw---- root:render 261,0` with `dank` in `render`, so the quadlet law is
+  `AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups` — the ComfyUI `/dev/kfd`
+  pattern, no new udev rule.
+- **Three premises of the issue corrected.** (1) The 13–16 % is the *vendor's* A/B
+  against *their* previous boot line, which our record never identified — we
+  copied the figure on 2026-09-11 without ever measuring here. Strict flush and
+  bounce buffering are not the mechanism either (Ubuntu ships lazy DMA, and a
+  coherent iGPU on unified memory takes no SWIOTLB bounce). What identity mode
+  actually costs the iGPU is the 1:1 page walk plus **ATS/PRI/PASID/GCR3** being
+  switched on at attach — the same four the PerfOpt enable path switches back off
+  (the `skip_caps` label in `attach_device`), and why it arms only "when an
+  integrated GPU is already in identity mode". (2) The NPU does **not** have to
+  wait for 7.4: `iommu=pt` enables the IOMMU, which is all `err=-19` asks for.
+  Making passthrough the AMD default was rejected on the list (Robin Murphy,
+  2026-03-26), so `pt` stays the sanctioned route. (3) The 460 tok/s baseline is
+  stale — the 2026-10-17 native-v2 payload runs **1,513–1,604 t/s cold prefill** —
+  so the window measures `off` vs `pt` locally and inherits no number.
+- Silicon check: our IOMMU is `1022:1508` Strix/Strix Halo rev 02, and Phoronix's
+  **Strix Halo desktop** arm measured only **2–4 %** from PerfOpt (18–23 % on
+  Strix Point / Krackan laptops) — the modest arm is the one that describes this
+  box. Kernel pick: **skip 6.18–6.18.7**, where an IOMMU regression disables SVA
+  outright.
+- NPU legality under identity mode: `amdxdna` binds via `iommu_sva_bind_device()`
+  + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain, independent of the default
+  domain type (`drivers/accel/amdxdna/aie2_pci.c`); `err=-19` needs only an
+  *enabled* IOMMU, which `pt` gives.
+- Self-correction inside the same session: the first pass — and the first version
+  of the posted comment — claimed `iommu=pt` alone already avoids the cost. It
+  does not. Patch v2 1/2 shows an identity attach keeps the page walk and enables
+  ATS/PRI/PASID/GCR3, so `pt` recovers **most** of the gap and PerfOpt is the last
+  part of it. The comment was edited to the corrected text.
+- Artifacts: new section in
+  `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` (+6 citations,
+  front-matter re-keyed to "vendor-claimed, never measured locally"),
+  `frank-npu.md` § *Why it is retired* corrected — the "`pt` is not a middle
+  ground" law is dead — the AGENTS.md key-lesson bullet re-keyed, a 4-step
+  maintenance-window procedure that measures locally instead of inheriting the
+  vendor number, and the plan posted to Gitea issue #3 (comment 23, then edited).
+  bats 371/371; commits `7643381` plus this correction. Nothing deployed — no
+  service was touched.