serverAppConfigs/serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
research	Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)	Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path.	dankclaw iommu kernel grub prefill frank-dragoon halogen frank-npu	2026-09-11

Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)

Current state (since 2026-10-18 01:24): Dankclaw boots iommu=pt — identity DMA, IOMMU armed, XDNA2 NPU bound and usable. Everything below is the record of the 2026-09-11 decision that booted it with amd_iommu=off on a vendor-claimed 13–16 % prefill figure, what that cost, and the measured flip back: see § 2026-10-18 execution at the bottom for the numbers and the revert path.

What changed

GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub on Dankclaw:

# before
ro quiet splash iommu=pt crashkernel=…

# after
ro quiet splash amd_iommu=off crashkernel=…

Then sudo update-grub and a reboot. Nothing else on the command line was touched.

Evidence

The upstream project published the measurement, and it is the only A/B they ran: 460 → 385 tok/s prefill at a 2,048-token prompt, 13 to 16 percent, taken on a Ryzen AI Max+ 395 with 128 GB — the same SKU as Dankclaw. Their published serving profile boots this way, so every number in their README is a number with translation off. Decode is unaffected; prefill is where the loss is.

The mechanism offered is DMA translation on the path every GPU allocation takes. The project states plainly that this is their configuration and not a tuning guide, that the flag turns off DMA translation machine-wide, and that it takes the NPU with it.

What it costs

1. The NPU endpoint dies. With IOMMU off there is no PASID/SVA state and /dev/accel/accel0 cannot be opened — err=-19, the exact failure documented in frank-npu. Audio input left the fleet with it; image input survives on frank-dragoon through halogen's vision tower.
2. Posture change machine-wide. Dankclaw also serves Caddy (the TLS edge and the Anubis/Authentik guard chain), Authentik, Pi-hole, Gitea, and the observability stack. DMA translation is now off for all of them. This was accepted explicitly as the price of the prefill figure.

What was deliberately NOT copied

The reference machine's command line has six settings. Only one was measured, so only that one was taken:

setting	reference value	why not here
amd_iommu=off	set	taken — the measured 13–16 %
amdgpu.gttsize	126976 MiB	a size, not a constant: it must equal installed RAM (their table: 128 GB → 126976, 96 GB → 95232, 64 GB → 63488). Dankclaw's GTT already reports 114 GiB ≈ 94 % of its 121 GiB without the flag, so it would buy nothing
ttm.pages_limit	32505856	the same number in 4 KiB pages (their 128 GB row is 99.3 % of RAM). Pasting it onto a different machine asks the driver for more GTT than exists
amdgpu.vm_update_mode=0	set	upstream has never run without it, makes no claim about what it buys, and lists it "for completeness rather than recommended"
amdgpu.noretry=0	set	as above
amdgpu.sg_display=0	set	as above

Verification and revert

cat /proc/cmdline                        # expect iommu=pt, no amd_iommu=off
ls /sys/kernel/iommu_groups/ | wc -l     # expect 32 (0 means the flip is gone)
cat /sys/kernel/iommu_groups/*/type | sort | uniq -c   # 31 identity + 1 DMA (the SD reader)
sudo dmesg | grep -m1 amdxdna            # driver bound + firmware loaded = NPU alive
curl -s http://127.0.0.1:7770/health     # frank-dragoon must still answer 200

To revert: set GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=off" in the tracked dankclaw/host/grub and run ./scripts/set-dankclaw-bootline.sh --reboot on Dankclaw. Never hand-edit /etc/default/grub: the tracked file is the source, and the installer diffs it, installs it 0644 root:root, runs update-grub, and checks the default grub.cfg entry before it will reboot. The IOMMU change is independent of the halogen deployment — reverting one does not require reverting the other.

2026-10-18 research: the penalty now has a hardware fix (AMD PerfOpt)

Asked and answered the same day the flip was executed (§ 2026-10-18 execution below): the figure behind the September decision is on an expiring path, and — the part that actually mattered — it was never measured on this host.

Mechanism. AMD IOMMU PerfOpt, spec-defined in the AMD I/O Virtualization Technology (IOMMU) spec §3.4.9: MMIO 0x016C bit 13 PERF_OPT_EN, gated by extended-feature bit 45 FEATURE_PERF_OPT. Armed, it stops GPA→SPA translation for the privileged integrated GPU — only the IR/IW permission bits are enforced — while the IOMMU stays fully enabled for every other device, the NPU included. Driver: drivers/iommu/amd/{iommu.c,init.c,amd_iommu_types.h}. The only consumer is amdgpu: amdgpu_device_use_perfopt() requires AMD_IS_APU and the identity domain, and the shared per-IOMMU bit is refcounted so one device's teardown cannot clear it under a peer.

Status. v2 merged through the iommu tree 2026-09-24, present in linux-next-20261016, queued for Linux 7.4 (merge window late October 2026, stable ~end of year, likely LTS candidate). NOT available here: /boot/ System.map-7.0.0-38-generic contains zero perfopt symbols, and Ubuntu 26.04's archives cap at 7.0.0-38.38 for both linux-image-generic and linux-generic-hwe-26.04-edge — reaching 7.4 means a mainline PPA or a local build. A 2026-09-28 follow-up removes the amdgpu.iommu_perfopt knob and makes PerfOpt compulsory.

Measured. Phoronix (2026-09-29, Lemonade + llama.cpp, same kernel, amdgpu.iommu_perfopt=0 vs default): 18–23 % on Strix Point / Krackan laptops, 2–4 % on a Strix Halo desktop — which is also the evidence that our silicon implements the bit (our IOMMU is 1022:1508 Strix/Strix Halo rev 02).

Identity mode is not "no IOMMU". Ubuntu ships CONFIG_IOMMU_DEFAULT_DMA_LAZY=y (STRICT not set), so IOTLB invalidation was already batched, and a coherent iGPU on unified memory takes no SWIOTLB bounce — strict flush and bounce buffering do not explain the September figure. But a device attached to the identity domain still has its DMA walked through 1:1 page tables, and the AMD attach path enables ATS/PRI/PASID/GCR3 for it; PerfOpt's own enable path has to switch those four off again (the skip_caps label in attach_device) to get the speedup, and the cover letter says it arms only "when an integrated GPU is already in identity mode". So pt is most of the recovery and PerfOpt is the last part of it.

The NPU is compatible with identity mode. amdxdna binds through iommu_sva_bind_device() + PASID + GCR3 into its own IOPT_TYPE_SVA domain, independent of the default domain type (drivers/accel/amdxdna/aie2_pci.c), and PerfOpt is not on that path at all. It only needs the IOMMU enabled — that is the whole err=-19 story. Kernel pick notes: the Ubuntu 6.17 -95 SVA failures were a GCR3 regression fixed before our kernel, but 6.18–6.18.7 disables SVA outright; skip that range.

Container access needs no reboot to settle: /dev/accel/accel0 already exists as crw-rw---- root:render 261,0 and dank is in render, so a quadlet wants AddDevice=/dev/accel/accel0 plus GroupAdd=keep-groups — the same pattern ComfyUI uses for /dev/kfd.

Revised procedure for the next boot-line window (still a maintenance window, still an explicit OK per run):

1. iommu=pt in place of amd_iommu=off. The NPU becomes usable at that reboot; the iGPU pays the identity-walk cost until PerfOpt exists on this kernel, which the desktop-class measurement prices in single-digit percent.
2. Measure off vs pt on this host, on the current payload (bench law: stop
   • mask frank-dragoon before any GPU A/B). The 13–16 % was the vendor's A/B against their previous boot line, which our record never identified — the 2026-09-11 adoption copied their figure without a local run. The current native-v2 payload does 1,513–1,604 t/s cold prefill, so that baseline is not comparable regardless.
3. Keep the fabric-clock pin (pp_dpm_fclk level 7) for GPU+NPU co-run.
4. On 7.4: nothing to do — PerfOpt arms itself when the iGPU is in the identity domain. Confirm with dmesg | grep -i perfopt; force off with amdgpu.iommu_perfopt=0 while the knob still exists.

2026-10-18 execution: the flip, measured on this host

Executed 01:24–01:35 with Frank's go-ahead. GRUB_CMDLINE_LINUX_DEFAULT went from quiet splash amd_iommu=off to quiet splash iommu=pt through the now-tracked dankclaw/host/grub + scripts/set-dankclaw-bootline.sh (diff → install 0644 root:root → update-grub → verify the default grub.cfg entry → reboot only on --reboot).

The A/B, on byte-identical traffic. The corpus is the rebuilt wikitext-2 raw test split — 2,891 lines, 1,287,656 B, the same file the pre-reboot boot read, and the engine confirms it by billing the same tokens (59,024 prefill prompt, 21,568 needle haystack). Both arms ran battery-frank-dragoon.py … tail against the live service with nothing else on the iGPU; the pt arm pre-warmed the 51.2 GB n-gram table with a full read (40 s) so a cold page cache could not masquerade as IOMMU cost.

probe	amd_iommu=off	iommu=pt
cold prefill, 59,024 tokens	1,585.1 t/s	1,616.0 t/s (+2.0 %)
warm prefill (prompt cache)	208,531 t/s	214,131 t/s
cache ratio	131.6×	132.5×
needle hi/lo @ 21,568 tokens	hit / hit	hit / hit
decode, 512 sampled tokens	37.3 t/s	33.7 t/s

Prefill cost: none. Identity mode is not slower here — it is +2.0 %, inside run-to-run spread. The single decode sample moved −9.7 %, and that instrument cannot see anything: three immediate repeats on the same boot gave 31.8, 46.9 and 47.8 t/s on identical requests, so any single-sample decode delta is noise. The 13–16 % that justified amd_iommu=off for five weeks does not exist on this host, and PerfOpt is now a bonus nobody is waiting for rather than the reason to flip.

Post-boot gates. 0 failed units, systemd running. 32 IOMMU groups — 31 identity + 1 DMA, and that one belongs to the Genesys Logic SD card reader (c2:00.0, a kernel quirk, not a fleet device). The iGPU (c5:00.0) sits in group 20 identity with ATS capability; interrupt remapping is enabled. frank-dragoon's boot ledger is unchanged to the MiB — 62.9 GiB weights (0.8 of them vision) + 5.5 GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all answered 200. (Later the same day the pool DOUBLING to 401,408 positions for slot parity was COMMITTED (2b12716) and deployed the same evening: the boot now reserves 401408 positions (about 11.0 GiB) and the engine-side total is 79.6 GiB in all (62.9 weights + 11.0 KV + 5.7 working), see frank-dragoon.)

The NPU came back with the flip. amdxdna bound to 0000:c6:00.1 (Strix/ Krackan/Strix Halo NPU rev 11), firmware amdnpu/17f0_11/npu_7.sbin, Initialized amdxdna_accel_driver 0.7.0 … on minor 0, IOMMU group 26 identity, PASID cap width 10 (Exec- Priv-). /dev/accel/accel0 is crw-rw---- root:render 261,0 and dank is in render, so a quadlet needs AddDevice=/dev/accel/accel0 + GroupAdd=keep-groups. No NPU endpoint is deployed — frank-npu stays the history of the retired endpoint; what changed is only its precondition.

Deliberately deferred: the pp_dpm_fclk pin (level 7, 2000 MHz) from the issue's co-run step. Pinning the fabric clock changes iGPU timing for every workload, and nothing co-runs on the NPU yet; it goes in when an NPU endpoint actually exists. PerfOpt still needs Linux 7.4 — 7.0.0-38-generic has zero perfopt symbols and the 26.04 archive caps at 7.0.0-38.38.

Tooling learned the hard way: /tmp is tmpfs, so the reboot deleted the eval stage mid-A/B. scripts/stage-frank-dragoon-eval.sh rebuilds it byte-identically (corpus + probe PNG + battery), and must run through a login shell because uv lives in ~/.local/bin.

Related

• frank-npu — the endpoint this retired, including the err=-19 root cause
• frank-dragoon — the runtime bought by the prefill figure
• frank-dragoon — the runtime it replaced (rollback target)

Citations

• peonist-ai/halogen-flash-server — README "The host settings these numbers were measured on", CHANGELOG 0.5.5–0.5.8
• iommu/amd: Add PerfOpt IOMMU performance optimization support (v2) — series 0/2 at /153378.html, merged via the iommu tree 2026-09-24 (/153844.html)
• Compulsory-PerfOpt follow-up (removes amdgpu.iommu_perfopt)
• Phoronix: AMD Boosting AI/LLM Performance For Radeon iGPUs As Much As 18~23% With Linux 7.4 — Strix Halo desktop arm (2–4 %), amdgpu.iommu_perfopt=0 A/B; news write-up here
• Robin Murphy rejecting the AMD passthrough-default patch — CONFIG_IOMMU_DEFAULT_PASSTHROUGH stays the sanctioned path
• Debian wiki: XDNA — driver + firmware upstreamed in 6.14; amdxdna broken on 6.18–6.18.7 by an IOMMU regression that disables SVA (report)
• amd/xdna-driver#704 — the err=-19 bind failure when the IOMMU is disabled