Blocker 2: pp_dpm_fclk at auto (levels to 2000 MHz); concurrent GPU+NPU hangs/corrupts until pinned to level 7 (deploy/host/halogen-fabric-clock unit, ~1 W).
Cost: IOMMU translation ON = 13–16 % prefill regression on frank-dragoon (upstream A/B; local measurement required after reboot). The NPU does NOT help custom voice (no ONNX/clone path exists) — this issue is for the small-model offload + a future NPU-Kokoro preset narrator only.
Scope
Maintenance window: stop+mask frank-dragoon/frank-zealot per benchmarking policy.
grub: amd_iommu=on (+ iommu=pt), reboot.
Install fabric-clock unit; verify the pin survives boot.
frank-dragoon quadlet: --device /dev/accel/accel0 + HALOGEN_NPU_MODELS per the models.txt catalog.
Run the image's own sweep; record prefill A/B vs the 460 tok/s baseline.
Acceptance
Sweep green; an NPU catalog model answers via the main port; prefill regression measured and recorded in the serverAppConfigs runbook; trunk decode within the published band.
> Moved from dank/command-center#4 (filed here by mistake of repo — this is fleet infra, not agent config).
Reboot-class change to a PRODUCTION box; needs a maintenance window + explicit OK per run. Census 2026-10-17.
## State measured
- `amdxdna` loaded, `/dev/accel/accel0` present, XRT 2.21 packages installed; frank-dragoon's pinned image HAS the `HALOGEN_NPU_MODELS` gate (models.txt tasks: decision/embedding/score/classify/generate — **no TTS task**).
- Blocker 1: `amd_iommu=off` on kernel cmdline (`iommu_groups==0`) → grub edit + reboot.
- Blocker 2: `pp_dpm_fclk` at auto (levels to 2000 MHz); concurrent GPU+NPU hangs/corrupts until pinned to level 7 (`deploy/host/halogen-fabric-clock` unit, ~1 W).
- Cost: IOMMU translation ON = 13–16 % prefill regression on frank-dragoon (upstream A/B; local measurement required after reboot). The NPU does NOT help custom voice (no ONNX/clone path exists) — this issue is for the small-model offload + a future NPU-Kokoro preset narrator only.
## Scope
1. Maintenance window: stop+mask frank-dragoon/frank-zealot per benchmarking policy.
2. grub: `amd_iommu=on` (+ `iommu=pt`), reboot.
3. Install fabric-clock unit; verify the pin survives boot.
4. frank-dragoon quadlet: `--device /dev/accel/accel0` + `HALOGEN_NPU_MODELS` per the models.txt catalog.
5. Run the image's own sweep; record prefill A/B vs the 460 tok/s baseline.
## Acceptance
Sweep green; an NPU catalog model answers via the main port; prefill regression measured and recorded in the serverAppConfigs runbook; trunk decode within the published band.
Research update (2026-10-18): the IOMMU blocker now has a hardware fix — but not on this kernel
Question: does a recent kernel change remove the translation penalty that justifies amd_iommu=off? Yes — and it decouples the two things this issue couples. Nothing was changed on Dankclaw today; the kernel that carries the fix is not installable from the Ubuntu 26.04 archive.
What changed upstream
AMD PerfOpt merged through the iommu tree 2026-09-24, first release Linux 7.4 (merge window late Oct 2026, stable ~end of year; present in linux-next-20261016). Spec §3.4.9: MMIO offset 0x016C bit 13 PERF_OPT_EN, gated by extended-feature bit 45 FEATURE_PERF_OPT.
Effect, in the patch's own words: it "allows privileged integrated I/O devices (GPUs) to bypass the IOMMU when directly accessing system memory. The IOMMU only enforces the IR/IW permission bits without GPA→SPA translations." The IOMMU stays fully armed for every other device — the NPU included. Driver: drivers/iommu/amd/{init.c,iommu.c,amd_iommu_types.h}; the bit is per-IOMMU and reference-counted, re-armed after suspend/resume, cleared on every teardown path.
PerfOpt is not an alternative to passthrough — it is its companion.amd_iommu_enable_perfopt() "performs a detach/reattach cycle to rehome devices already on the identity domain" with ATS/PRI/PASID/GCR3 disabled (the skip_caps label in attach_device), and the cover letter says amdgpu turns it on "when an integrated GPU is already in identity mode". A 2026-09-28 follow-up makes it compulsory and deletes the amdgpu.iommu_perfopt knob.
Measured (Phoronix 2026-09-29, Lemonade + llama.cpp, same kernel, amdgpu.iommu_perfopt=0 vs default): 18–23 % on Strix Point / Krackan laptops, 2–4 % on a Strix Halo desktop. Our silicon implements the bit — IOMMU is 1022:1508 Strix/Strix Halo rev 02 — and the desktop arm is the one that describes this box.
The 13–16 % was never measured on this host, and its mechanism was mis-stated. It is the vendor's A/B against their previous boot line — our record never identified which — and we adopted it on 2026-09-11 without a local run. It is also not strict-flush or bounce cost: Ubuntu ships lazy (batched) IOTLB invalidation and a coherent iGPU on unified memory takes no SWIOTLB bounce. The real identity-mode cost is the 1:1 page walk plus ATS/PRI/PASID/GCR3 being switched on at attach — precisely the four things PerfOpt switches back off.
iommu=pt is most of the fix, not all of it, and the NPU does not have to wait for Linux 7.4.pt gives the enabled IOMMU that err=-19 is about, so the NPU returns at that reboot; the iGPU still pays the identity walk until PerfOpt exists on this kernel, which on this silicon class is single-digit percent (the 2–4 % desktop arm). Making passthrough the AMD default was rejected on the list (Robin Murphy, 2026-03-26: CONFIG_IOMMU_DEFAULT_PASSTHROUGH stays the sanctioned route), so pt is the legitimate way to get there.
The 460 tok/s baseline quoted above is stale. That is the 2026-09-11 halogen shape; the 2026-10-17 native-v2 payload measures 1,513–1,604 t/s cold prefill at ~59k tokens. The window must measure off vs pt here and now, not inherit any published figure.
Why the NPU is legal under identity mode
amdxdna binds with iommu_sva_bind_device() + PASID + GCR3 into its own IOPT_TYPE_SVA domain, independent of the default domain type (drivers/accel/amdxdna/aie2_pci.c). PerfOpt is not on that path at all — and note the asymmetry worth keeping straight: PerfOpt disables ATS/PRI/PASID/GCR3 for the iGPU, while the NPU needs PASID/GCR3 for itself. Different devices, different domains, no conflict. Kernel pick: skip 6.18–6.18.7, where an IOMMU regression disables SVA outright; Ubuntu's 6.17 -95 GCR3 breakage predates our kernel.
Container access — settled without a reboot
/dev/accel/accel0 already exists and is group-writable to render, and dank is in render, so the endpoint needs a quadlet with AddDevice=/dev/accel/accel0 plus GroupAdd=keep-groups — the same pattern ComfyUI uses for /dev/kfd. No new udev rule required.
Corrected procedure for the maintenance window
iommu=pt in place of amd_iommu=off in GRUB_CMDLINE_LINUX_DEFAULT, sudo update-grub, reboot. Gate: IOMMU group count > 0, lspci -k names the amd_iommu driver, frank-dragoon answers /health 200.
Measure off vs pt on this host, on the current v2 payload (bench law: stop + mask frank-dragoon first — never bench beside live traffic, and a mask cannot be applied to a relinked unit, so re-deploy afterwards instead of unmasking). Accept the flip only with the cost written down.
Pin the fabric clock (pp_dpm_fclk level 7) for GPU+NPU co-run.
Bring the NPU up as a boot-persistent quadlet with the device flags above.
Later, on Linux 7.4 (mainline PPA or local build — nothing in the 26.04 archive): nothing to do, PerfOpt arms itself once the iGPU sits in the identity domain. Confirm with dmesg | grep -i perfopt; force off with amdgpu.iommu_perfopt=0 while that knob still exists.
Documented in serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md (§ 2026-10-18 research: the penalty now has a hardware fix), with frank-npu.md § Why it is retired corrected — the "iommu=pt is not a middle ground" law recorded there is dead — and the AGENTS.md key-lesson bullet re-keyed. Committed on main as 7643381 plus a follow-up correction commit.
## Research update (2026-10-18): the IOMMU blocker now has a hardware fix — but not on this kernel
Question: does a recent kernel change remove the translation penalty that justifies `amd_iommu=off`? **Yes — and it decouples the two things this issue couples.** Nothing was changed on Dankclaw today; the kernel that carries the fix is not installable from the Ubuntu 26.04 archive.
### What changed upstream
- **AMD PerfOpt** merged through the iommu tree **2026-09-24**, first release **Linux 7.4** (merge window late Oct 2026, stable ~end of year; present in `linux-next-20261016`). Spec §3.4.9: MMIO offset `0x016C` bit 13 `PERF_OPT_EN`, gated by extended-feature bit 45 `FEATURE_PERF_OPT`.
- Effect, in the patch's own words: it "allows privileged integrated I/O devices (GPUs) to bypass the IOMMU when directly accessing system memory. The IOMMU only enforces the IR/IW permission bits without GPA→SPA translations." The IOMMU stays fully armed for every other device — **the NPU included**. Driver: `drivers/iommu/amd/{init.c,iommu.c,amd_iommu_types.h}`; the bit is per-IOMMU and **reference-counted**, re-armed after suspend/resume, cleared on every teardown path.
- **PerfOpt is not an alternative to passthrough — it is its companion.** `amd_iommu_enable_perfopt()` "performs a detach/reattach cycle to rehome devices **already on the identity domain**" with **ATS/PRI/PASID/GCR3 disabled** (the `skip_caps` label in `attach_device`), and the cover letter says amdgpu turns it on "when an integrated GPU is **already in identity mode**". A 2026-09-28 follow-up makes it compulsory and deletes the `amdgpu.iommu_perfopt` knob.
- Measured (Phoronix 2026-09-29, Lemonade + llama.cpp, same kernel, `amdgpu.iommu_perfopt=0` vs default): **18–23 % on Strix Point / Krackan laptops, 2–4 % on a Strix Halo desktop**. Our silicon implements the bit — IOMMU is `1022:1508` Strix/Strix Halo rev 02 — and the desktop arm is the one that describes this box.
### Local census (read-only, no reboot)
| probe | result |
|---|---|
| `uname -r` | `7.0.0-38-generic` |
| `/proc/cmdline` | `amd_iommu=off` present |
| IOMMU group count | 0 |
| `perfopt` symbols in `System.map-7.0.0-38-generic` | **0** |
| `linux-image-generic`, `linux-generic-hwe-26.04-edge` | both pinned `7.0.0-38.38` — no 7.4 in the archive |
| `CONFIG_IOMMU_DEFAULT_DMA_LAZY` / `_STRICT` / `_PASSTHROUGH` | `y` / unset / unset |
| `/dev/accel/accel0` | exists, `crw-rw---- root:render` 261,0; `dank` ∈ `render` |
### Three corrections to this issue's premise
1. **The 13–16 % was never measured on this host, and its mechanism was mis-stated.** It is the vendor's A/B against *their* previous boot line — our record never identified which — and we adopted it on 2026-09-11 without a local run. It is also not strict-flush or bounce cost: Ubuntu ships lazy (batched) IOTLB invalidation and a coherent iGPU on unified memory takes no SWIOTLB bounce. The real identity-mode cost is the **1:1 page walk plus ATS/PRI/PASID/GCR3 being switched on at attach** — precisely the four things PerfOpt switches back off.
2. **`iommu=pt` is most of the fix, not all of it, and the NPU does not have to wait for Linux 7.4.** `pt` gives the enabled IOMMU that `err=-19` is about, so the NPU returns at that reboot; the iGPU still pays the identity walk until PerfOpt exists on this kernel, which on this silicon class is single-digit percent (the 2–4 % desktop arm). Making passthrough the AMD default was rejected on the list (Robin Murphy, 2026-03-26: `CONFIG_IOMMU_DEFAULT_PASSTHROUGH` stays the sanctioned route), so `pt` is the legitimate way to get there.
3. **The 460 tok/s baseline quoted above is stale.** That is the 2026-09-11 halogen shape; the 2026-10-17 native-v2 payload measures **1,513–1,604 t/s cold prefill** at ~59k tokens. The window must measure `off` vs `pt` here and now, not inherit any published figure.
### Why the NPU is legal under identity mode
`amdxdna` binds with `iommu_sva_bind_device()` + PASID + GCR3 into its own `IOPT_TYPE_SVA` domain, **independent of the default domain type** (`drivers/accel/amdxdna/aie2_pci.c`). PerfOpt is not on that path at all — and note the asymmetry worth keeping straight: PerfOpt *disables* ATS/PRI/PASID/GCR3 for the **iGPU**, while the NPU *needs* PASID/GCR3 for itself. Different devices, different domains, no conflict. Kernel pick: **skip 6.18–6.18.7**, where an IOMMU regression disables SVA outright; Ubuntu's 6.17 `-95` GCR3 breakage predates our kernel.
### Container access — settled without a reboot
`/dev/accel/accel0` already exists and is group-writable to `render`, and `dank` is in `render`, so the endpoint needs a quadlet with `AddDevice=/dev/accel/accel0` plus `GroupAdd=keep-groups` — the same pattern ComfyUI uses for `/dev/kfd`. No new udev rule required.
### Corrected procedure for the maintenance window
1. `iommu=pt` in place of `amd_iommu=off` in `GRUB_CMDLINE_LINUX_DEFAULT`, `sudo update-grub`, reboot. Gate: IOMMU group count > 0, `lspci -k` names the `amd_iommu` driver, frank-dragoon answers `/health` 200.
2. **Measure `off` vs `pt` on this host, on the current v2 payload** (bench law: stop + mask `frank-dragoon` first — never bench beside live traffic, and a mask cannot be applied to a relinked unit, so re-deploy afterwards instead of unmasking). Accept the flip only with the cost written down.
3. Pin the fabric clock (`pp_dpm_fclk` level 7) for GPU+NPU co-run.
4. Bring the NPU up as a boot-persistent quadlet with the device flags above.
5. Later, on Linux 7.4 (mainline PPA or local build — nothing in the 26.04 archive): nothing to do, PerfOpt arms itself once the iGPU sits in the identity domain. Confirm with `dmesg | grep -i perfopt`; force off with `amdgpu.iommu_perfopt=0` while that knob still exists.
Documented in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` (§ *2026-10-18 research: the penalty now has a hardware fix*), with `frank-npu.md` § *Why it is retired* corrected — the "`iommu=pt` is not a middle ground" law recorded there is dead — and the AGENTS.md key-lesson bullet re-keyed. Committed on main as `7643381` plus a follow-up correction commit.
Executed 2026-10-18 01:24–01:35: the host boots iommu=pt, and the penalty measured zero
Window ran on Frank's go-ahead, including the local A/B this issue was missing.
The A/B (byte-identical traffic, live service, nothing else on the iGPU)
probe
amd_iommu=off (before)
iommu=pt (after)
cold prefill, 59,024 tokens
1,585.1 t/s
1,616.0 t/s (+2.0 %)
warm prefill (prompt cache)
208,531 t/s
214,131 t/s
cache ratio
131.6×
132.5×
needle hi/lo @ 21,568 tokens
hit / hit
hit / hit
decode, 512 sampled tokens
37.3 t/s
33.7 t/s
Fairness controls: the corpus is the rebuilt wikitext-2 raw test split — 2,891 lines, 1,287,656 B — and the engine proves the inputs match by billing identical token counts (59,024 / 21,568) on both boots; the pt arm pre-read the 51.2 GB n-gram table (40 s) so a cold page cache could not be mistaken for IOMMU cost.
Conclusion: identity DMA costs nothing on this host. The 13–16 % that justified amd_iommu=off since 2026-09-11 was never measured here and does not appear when measured. The decode delta is an instrument artifact, not a result: three repeats of the same request on the same boot gave 31.8, 46.9 and 47.8 t/s, so any single-sample decode number is noise on this probe. PerfOpt remains a real (2–4 %) win on this silicon class, but nobody is waiting for it now.
NPU: precondition satisfied, endpoint still not deployed
amdxdna bound 0000:c6:00.1 (Strix/Krackan/Strix Halo NPU rev 11), firmware amdnpu/17f0_11/npu_7.sbin, Initialized amdxdna_accel_driver 0.7.0 … on minor 0. IOMMU group 26 identity, PASID cap width 10 (Exec- Priv-). /dev/accel/accel0 is crw-rw---- root:render 261,0 and dank is in render, so the revival quadlet needs AddDevice=/dev/accel/accel0 + GroupAdd=keep-groups — no new udev rule. What remains on this issue is the endpoint itself (runtime + quadlet + model), not the boot line.
Fleet state after the reboot
0 failed units, systemd running. 32 IOMMU groups: 31 identity + 1 DMA, and that one is the Genesys Logic SD card reader (c2:00.0) — a kernel quirk, not a fleet device. iGPU c5:00.0 sits in group 20 identity with ATS; interrupt remapping enabled. frank-dragoon's ledger is unchanged to the MiB (62.9 GiB weights + 5.5 KV + 5.7 working = 74.1 GiB, 2 slots over ONE 200,704 pool, MTP depth 2, prompt cache ON, no clamp; listening 39.8 s after start) and Gitea / the Caddy edge / Grafana answered 200.
Deliberately not done
pp_dpm_fclk level-7 pin: it changes iGPU timing for every workload and nothing co-runs on the NPU yet. Goes in when an NPU endpoint exists — step 3 stays open here.
PerfOpt: needs Linux 7.4; 7.0.0-38-generic has zero perfopt symbols and the 26.04 archive pins 7.0.0-38.38.
Tooling and docs
/etc/default/grub is now tracked (dankclaw/host/grub) with scripts/set-dankclaw-bootline.sh as its only door (diff → install → update-grub → verify the default grub.cfg entry → reboot on --reboot). scripts/stage-frank-dragoon-eval.sh rebuilds the eval stage, because /tmp is tmpfs and the reboot ate it mid-A/B. Recorded in serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md § 2026-10-18 execution; frank-npu.md re-keyed. Commits 1b373f3 (flip) and 34a30d8 (record) on main, bats 371/371.
## Executed 2026-10-18 01:24–01:35: the host boots `iommu=pt`, and the penalty measured **zero**
Window ran on Frank's go-ahead, including the local A/B this issue was missing.
### The A/B (byte-identical traffic, live service, nothing else on the iGPU)
| probe | `amd_iommu=off` (before) | `iommu=pt` (after) |
|---|---|---|
| cold prefill, 59,024 tokens | 1,585.1 t/s | **1,616.0 t/s (+2.0 %)** |
| warm prefill (prompt cache) | 208,531 t/s | 214,131 t/s |
| cache ratio | 131.6× | 132.5× |
| needle hi/lo @ 21,568 tokens | hit / hit | hit / hit |
| decode, 512 sampled tokens | 37.3 t/s | 33.7 t/s |
Fairness controls: the corpus is the rebuilt wikitext-2 raw test split — 2,891 lines, **1,287,656 B** — and the engine proves the inputs match by billing identical token counts (59,024 / 21,568) on both boots; the `pt` arm pre-read the 51.2 GB n-gram table (40 s) so a cold page cache could not be mistaken for IOMMU cost.
**Conclusion: identity DMA costs nothing on this host.** The 13–16 % that justified `amd_iommu=off` since 2026-09-11 was never measured here and does not appear when measured. The decode delta is an instrument artifact, not a result: three repeats of the *same request on the same boot* gave 31.8, 46.9 and 47.8 t/s, so any single-sample decode number is noise on this probe. PerfOpt remains a real (2–4 %) win on this silicon class, but nobody is waiting for it now.
### NPU: precondition satisfied, endpoint still not deployed
`amdxdna` bound `0000:c6:00.1` (Strix/Krackan/Strix Halo NPU rev 11), firmware `amdnpu/17f0_11/npu_7.sbin`, `Initialized amdxdna_accel_driver 0.7.0 … on minor 0`. IOMMU group 26 `identity`, PASID cap width 10 (`Exec- Priv-`). `/dev/accel/accel0` is `crw-rw---- root:render 261,0` and `dank` is in `render`, so the revival quadlet needs `AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups` — no new udev rule. **What remains on this issue is the endpoint itself** (runtime + quadlet + model), not the boot line.
### Fleet state after the reboot
0 failed units, `systemd` running. 32 IOMMU groups: 31 `identity` + 1 `DMA`, and that one is the Genesys Logic SD card reader (`c2:00.0`) — a kernel quirk, not a fleet device. iGPU `c5:00.0` sits in group 20 `identity` with ATS; interrupt remapping enabled. frank-dragoon's ledger is unchanged to the MiB (62.9 GiB weights + 5.5 KV + 5.7 working = 74.1 GiB, 2 slots over ONE 200,704 pool, MTP depth 2, prompt cache ON, no clamp; listening 39.8 s after start) and Gitea / the Caddy edge / Grafana answered 200.
### Deliberately not done
- **`pp_dpm_fclk` level-7 pin**: it changes iGPU timing for every workload and nothing co-runs on the NPU yet. Goes in when an NPU endpoint exists — step 3 stays open here.
- **PerfOpt**: needs Linux 7.4; `7.0.0-38-generic` has zero `perfopt` symbols and the 26.04 archive pins `7.0.0-38.38`.
### Tooling and docs
`/etc/default/grub` is now tracked (`dankclaw/host/grub`) with `scripts/set-dankclaw-bootline.sh` as its only door (diff → install → `update-grub` → verify the default `grub.cfg` entry → reboot on `--reboot`). `scripts/stage-frank-dragoon-eval.sh` rebuilds the eval stage, because `/tmp` is tmpfs and the reboot ate it mid-A/B. Recorded in `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` § *2026-10-18 execution*; `frank-npu.md` re-keyed. Commits `1b373f3` (flip) and `34a30d8` (record) on main, bats 371/371.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Reboot-class change to a PRODUCTION box; needs a maintenance window + explicit OK per run. Census 2026-10-17.
State measured
amdxdnaloaded,/dev/accel/accel0present, XRT 2.21 packages installed; frank-dragoon's pinned image HAS theHALOGEN_NPU_MODELSgate (models.txt tasks: decision/embedding/score/classify/generate — no TTS task).amd_iommu=offon kernel cmdline (iommu_groups==0) → grub edit + reboot.pp_dpm_fclkat auto (levels to 2000 MHz); concurrent GPU+NPU hangs/corrupts until pinned to level 7 (deploy/host/halogen-fabric-clockunit, ~1 W).Scope
amd_iommu=on(+iommu=pt), reboot.--device /dev/accel/accel0+HALOGEN_NPU_MODELSper the models.txt catalog.Acceptance
Sweep green; an NPU catalog model answers via the main port; prefill regression measured and recorded in the serverAppConfigs runbook; trunk decode within the published band.
Research update (2026-10-18): the IOMMU blocker now has a hardware fix — but not on this kernel
Question: does a recent kernel change remove the translation penalty that justifies
amd_iommu=off? Yes — and it decouples the two things this issue couples. Nothing was changed on Dankclaw today; the kernel that carries the fix is not installable from the Ubuntu 26.04 archive.What changed upstream
linux-next-20261016). Spec §3.4.9: MMIO offset0x016Cbit 13PERF_OPT_EN, gated by extended-feature bit 45FEATURE_PERF_OPT.drivers/iommu/amd/{init.c,iommu.c,amd_iommu_types.h}; the bit is per-IOMMU and reference-counted, re-armed after suspend/resume, cleared on every teardown path.amd_iommu_enable_perfopt()"performs a detach/reattach cycle to rehome devices already on the identity domain" with ATS/PRI/PASID/GCR3 disabled (theskip_capslabel inattach_device), and the cover letter says amdgpu turns it on "when an integrated GPU is already in identity mode". A 2026-09-28 follow-up makes it compulsory and deletes theamdgpu.iommu_perfoptknob.amdgpu.iommu_perfopt=0vs default): 18–23 % on Strix Point / Krackan laptops, 2–4 % on a Strix Halo desktop. Our silicon implements the bit — IOMMU is1022:1508Strix/Strix Halo rev 02 — and the desktop arm is the one that describes this box.Local census (read-only, no reboot)
uname -r7.0.0-38-generic/proc/cmdlineamd_iommu=offpresentperfoptsymbols inSystem.map-7.0.0-38-genericlinux-image-generic,linux-generic-hwe-26.04-edge7.0.0-38.38— no 7.4 in the archiveCONFIG_IOMMU_DEFAULT_DMA_LAZY/_STRICT/_PASSTHROUGHy/ unset / unset/dev/accel/accel0crw-rw---- root:render261,0;dank∈renderThree corrections to this issue's premise
iommu=ptis most of the fix, not all of it, and the NPU does not have to wait for Linux 7.4.ptgives the enabled IOMMU thaterr=-19is about, so the NPU returns at that reboot; the iGPU still pays the identity walk until PerfOpt exists on this kernel, which on this silicon class is single-digit percent (the 2–4 % desktop arm). Making passthrough the AMD default was rejected on the list (Robin Murphy, 2026-03-26:CONFIG_IOMMU_DEFAULT_PASSTHROUGHstays the sanctioned route), soptis the legitimate way to get there.offvspthere and now, not inherit any published figure.Why the NPU is legal under identity mode
amdxdnabinds withiommu_sva_bind_device()+ PASID + GCR3 into its ownIOPT_TYPE_SVAdomain, independent of the default domain type (drivers/accel/amdxdna/aie2_pci.c). PerfOpt is not on that path at all — and note the asymmetry worth keeping straight: PerfOpt disables ATS/PRI/PASID/GCR3 for the iGPU, while the NPU needs PASID/GCR3 for itself. Different devices, different domains, no conflict. Kernel pick: skip 6.18–6.18.7, where an IOMMU regression disables SVA outright; Ubuntu's 6.17-95GCR3 breakage predates our kernel.Container access — settled without a reboot
/dev/accel/accel0already exists and is group-writable torender, anddankis inrender, so the endpoint needs a quadlet withAddDevice=/dev/accel/accel0plusGroupAdd=keep-groups— the same pattern ComfyUI uses for/dev/kfd. No new udev rule required.Corrected procedure for the maintenance window
iommu=ptin place ofamd_iommu=offinGRUB_CMDLINE_LINUX_DEFAULT,sudo update-grub, reboot. Gate: IOMMU group count > 0,lspci -knames theamd_iommudriver, frank-dragoon answers/health200.offvspton this host, on the current v2 payload (bench law: stop + maskfrank-dragoonfirst — never bench beside live traffic, and a mask cannot be applied to a relinked unit, so re-deploy afterwards instead of unmasking). Accept the flip only with the cost written down.pp_dpm_fclklevel 7) for GPU+NPU co-run.dmesg | grep -i perfopt; force off withamdgpu.iommu_perfopt=0while that knob still exists.Documented in
serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md(§ 2026-10-18 research: the penalty now has a hardware fix), withfrank-npu.md§ Why it is retired corrected — the "iommu=ptis not a middle ground" law recorded there is dead — and the AGENTS.md key-lesson bullet re-keyed. Committed on main as7643381plus a follow-up correction commit.Executed 2026-10-18 01:24–01:35: the host boots
iommu=pt, and the penalty measured zeroWindow ran on Frank's go-ahead, including the local A/B this issue was missing.
The A/B (byte-identical traffic, live service, nothing else on the iGPU)
amd_iommu=off(before)iommu=pt(after)Fairness controls: the corpus is the rebuilt wikitext-2 raw test split — 2,891 lines, 1,287,656 B — and the engine proves the inputs match by billing identical token counts (59,024 / 21,568) on both boots; the
ptarm pre-read the 51.2 GB n-gram table (40 s) so a cold page cache could not be mistaken for IOMMU cost.Conclusion: identity DMA costs nothing on this host. The 13–16 % that justified
amd_iommu=offsince 2026-09-11 was never measured here and does not appear when measured. The decode delta is an instrument artifact, not a result: three repeats of the same request on the same boot gave 31.8, 46.9 and 47.8 t/s, so any single-sample decode number is noise on this probe. PerfOpt remains a real (2–4 %) win on this silicon class, but nobody is waiting for it now.NPU: precondition satisfied, endpoint still not deployed
amdxdnabound0000:c6:00.1(Strix/Krackan/Strix Halo NPU rev 11), firmwareamdnpu/17f0_11/npu_7.sbin,Initialized amdxdna_accel_driver 0.7.0 … on minor 0. IOMMU group 26identity, PASID cap width 10 (Exec- Priv-)./dev/accel/accel0iscrw-rw---- root:render 261,0anddankis inrender, so the revival quadlet needsAddDevice=/dev/accel/accel0+GroupAdd=keep-groups— no new udev rule. What remains on this issue is the endpoint itself (runtime + quadlet + model), not the boot line.Fleet state after the reboot
0 failed units,
systemdrunning. 32 IOMMU groups: 31identity+ 1DMA, and that one is the Genesys Logic SD card reader (c2:00.0) — a kernel quirk, not a fleet device. iGPUc5:00.0sits in group 20identitywith ATS; interrupt remapping enabled. frank-dragoon's ledger is unchanged to the MiB (62.9 GiB weights + 5.5 KV + 5.7 working = 74.1 GiB, 2 slots over ONE 200,704 pool, MTP depth 2, prompt cache ON, no clamp; listening 39.8 s after start) and Gitea / the Caddy edge / Grafana answered 200.Deliberately not done
pp_dpm_fclklevel-7 pin: it changes iGPU timing for every workload and nothing co-runs on the NPU yet. Goes in when an NPU endpoint exists — step 3 stays open here.7.0.0-38-generichas zeroperfoptsymbols and the 26.04 archive pins7.0.0-38.38.Tooling and docs
/etc/default/grubis now tracked (dankclaw/host/grub) withscripts/set-dankclaw-bootline.shas its only door (diff → install →update-grub→ verify the defaultgrub.cfgentry → reboot on--reboot).scripts/stage-frank-dragoon-eval.shrebuilds the eval stage, because/tmpis tmpfs and the reboot ate it mid-A/B. Recorded inserverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md§ 2026-10-18 execution;frank-npu.mdre-keyed. Commits1b373f3(flip) and34a30d8(record) on main, bats 371/371.