docs(iommu): record the executed flip to iommu=pt and its measured cost of nothing
Window 01:24-01:35. Byte-identical A/B (rebuild proved it: same 1,287,656 B corpus, same billed token counts 59,024 / 21,568): cold prefill 1,585.1 -> 1,616.0 t/s (+2.0 %), warm 208,531 -> 214,131, needle hit/hit both boots. The 13-16 % that justified amd_iommu=off since 2026-09-11 does not exist on this host; the decode delta (-9.7 %) is instrument noise -- three repeats of one request on the same boot gave 31.8/46.9/47.8 t/s. NPU precondition restored: amdxdna bound 0000:c6:00.1 (driver 0.7.0, firmware amdnpu/17f0_11/npu_7.sbin), group 26 identity, PASID width 10, /dev/accel/accel0 root:render with dank in render. No endpoint deployed, so frank-npu stays retired -- only its rationale changes. Gates: 0 failed units, 32 groups (31 identity + the SD reader's DMA quirk), iGPU c5:00.0 identity with ATS, dragoon ledger unchanged (74.1 GiB, no clamp, listening at 39.8 s). Deferred on purpose: the pp_dpm_fclk pin (changes iGPU timing for everyone, nothing co-runs yet) and PerfOpt (7.0.0-38 has zero perfopt symbols, 26.04 caps at 7.0.0-38.38). Re-keyed: decision record banner/front matter/verification/revert + new execution section, frank-npu rationale, AGENTS key lesson, both index pages.
This commit is contained in:
Frank committed
1 parent
5ad28c19d8
commit
34a30d8505
6 files changed
+160
-48
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: research
|
||||
title: Dankclaw IOMMU translation off (amd_iommu=off) — decision record
|
||||
description: Why Dankclaw boots with amd_iommu=off since 2026-09-11, what it buys (a vendor-claimed 13-16% prefill for frank-dragoon, never measured locally), what it costs (the XDNA2 NPU endpoint, machine-wide DMA translation), the 2026-10-18 AMD PerfOpt research that puts the trade on an expiring path, and how to revert.
|
||||
title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)
|
||||
description: Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path.
|
||||
tags:
|
||||
- dankclaw
|
||||
- iommu
|
||||
@@ -16,10 +16,11 @@ timestamp: 2026-09-11
|
||||
|
||||
# Dankclaw IOMMU translation off — decision record
|
||||
|
||||
**Decision (2026-09-11):** Dankclaw boots with **`amd_iommu=off`** instead of
|
||||
`iommu=pt`, to recover the prefill throughput the large-model runtime loses to
|
||||
IOMMU translation. The XDNA2 NPU endpoint was accepted as the cost and is
|
||||
retired ([[frank-npu]]).
|
||||
**Current state (since 2026-10-18 01:24): Dankclaw boots `iommu=pt`** — identity
|
||||
DMA, IOMMU armed, XDNA2 NPU bound and usable. Everything below is the record of
|
||||
the 2026-09-11 decision that booted it with **`amd_iommu=off`** on a vendor-claimed
|
||||
13–16 % prefill figure, what that cost, and the measured flip back: see
|
||||
§ *2026-10-18 execution* at the bottom for the numbers and the revert path.
|
||||
|
||||
## What changed
|
||||
|
||||
@@ -77,23 +78,26 @@ so only that one was taken:
|
||||
## Verification and revert
|
||||
|
||||
```bash
|
||||
cat /proc/cmdline # expect amd_iommu=off, no iommu=pt
|
||||
ls /sys/kernel/iommu_groups/ | wc -l # expect 0 groups (was 30)
|
||||
systemctl is-active frank-npu # expect inactive (masked)
|
||||
cat /proc/cmdline # expect iommu=pt, no amd_iommu=off
|
||||
ls /sys/kernel/iommu_groups/ | wc -l # expect 32 (0 means the flip is gone)
|
||||
cat /sys/kernel/iommu_groups/*/type | sort | uniq -c # 31 identity + 1 DMA (the SD reader)
|
||||
sudo dmesg | grep -m1 amdxdna # driver bound + firmware loaded = NPU alive
|
||||
curl -s http://127.0.0.1:7770/health # frank-dragoon must still answer 200
|
||||
```
|
||||
|
||||
**To revert:** put `iommu=pt` back in `GRUB_CMDLINE_LINUX_DEFAULT` in place of
|
||||
`amd_iommu=off`, run `sudo update-grub`, reboot, and unmask/start `frank-npu`
|
||||
with the archived unit (`.AGENTTRASH/frank-npu.service.flm-npu`) if the NPU
|
||||
endpoint is wanted back. The IOMMU change is independent of the halogen
|
||||
deployment — reverting one does not require reverting the other.
|
||||
**To revert:** set `GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=off"` in
|
||||
the tracked `dankclaw/host/grub` and run `./scripts/set-dankclaw-bootline.sh
|
||||
--reboot` on Dankclaw. Never hand-edit `/etc/default/grub`: the tracked file is
|
||||
the source, and the installer diffs it, installs it 0644 root:root, runs
|
||||
`update-grub`, and checks the default `grub.cfg` entry before it will reboot. The
|
||||
IOMMU change is independent of the halogen deployment — reverting one does not
|
||||
require reverting the other.
|
||||
|
||||
## 2026-10-18 research: the penalty now has a hardware fix (AMD PerfOpt)
|
||||
|
||||
Asked and answered — **the decision changes, the host does not.** Nothing was
|
||||
changed on Dankclaw today; this section records why the figure behind the
|
||||
decision is on an expiring path — and that it was never measured on this host.
|
||||
Asked and answered the same day the flip was executed (§ *2026-10-18 execution*
|
||||
below): the figure behind the September decision is on an expiring path, and — the
|
||||
part that actually mattered — it was **never measured on this host**.
|
||||
|
||||
**Mechanism.** AMD IOMMU **PerfOpt**, spec-defined in the AMD I/O Virtualization
|
||||
Technology (IOMMU) spec §3.4.9: MMIO `0x016C` bit 13 `PERF_OPT_EN`, gated by
|
||||
@@ -159,6 +163,65 @@ still an explicit OK per run):
|
||||
domain. Confirm with `dmesg | grep -i perfopt`; force off with
|
||||
`amdgpu.iommu_perfopt=0` while the knob still exists.
|
||||
|
||||
## 2026-10-18 execution: the flip, measured on this host
|
||||
|
||||
Executed 01:24–01:35 with Frank's go-ahead. `GRUB_CMDLINE_LINUX_DEFAULT` went from
|
||||
`quiet splash amd_iommu=off` to `quiet splash iommu=pt` through the now-tracked
|
||||
`dankclaw/host/grub` + `scripts/set-dankclaw-bootline.sh` (diff → install 0644
|
||||
root:root → `update-grub` → verify the default `grub.cfg` entry → reboot only on
|
||||
`--reboot`).
|
||||
|
||||
**The A/B, on byte-identical traffic.** The corpus is the rebuilt wikitext-2 raw
|
||||
test split — 2,891 lines, **1,287,656 B**, the same file the pre-reboot boot read,
|
||||
and the engine confirms it by billing the same tokens (59,024 prefill prompt,
|
||||
21,568 needle haystack). Both arms ran `battery-frank-dragoon.py … tail` against
|
||||
the live service with nothing else on the iGPU; the pt arm pre-warmed the 51.2 GB
|
||||
n-gram table with a full read (40 s) so a cold page cache could not masquerade as
|
||||
IOMMU cost.
|
||||
|
||||
| probe | `amd_iommu=off` | `iommu=pt` |
|
||||
|---|---|---|
|
||||
| cold prefill, 59,024 tokens | 1,585.1 t/s | **1,616.0 t/s (+2.0 %)** |
|
||||
| warm prefill (prompt cache) | 208,531 t/s | 214,131 t/s |
|
||||
| cache ratio | 131.6× | 132.5× |
|
||||
| needle hi/lo @ 21,568 tokens | hit / hit | hit / hit |
|
||||
| decode, 512 sampled tokens | 37.3 t/s | 33.7 t/s |
|
||||
|
||||
**Prefill cost: none.** Identity mode is not slower here — it is +2.0 %, inside
|
||||
run-to-run spread. The single decode sample moved −9.7 %, and that instrument
|
||||
cannot see anything: three immediate repeats on the same boot gave 31.8, 46.9 and
|
||||
47.8 t/s on identical requests, so any single-sample decode delta is noise. The
|
||||
13–16 % that justified `amd_iommu=off` for five weeks does not exist on this host,
|
||||
and PerfOpt is now a bonus nobody is waiting for rather than the reason to flip.
|
||||
|
||||
**Post-boot gates.** 0 failed units, `systemd` running. 32 IOMMU groups — 31
|
||||
`identity` + 1 `DMA`, and that one belongs to the Genesys Logic SD card reader
|
||||
(`c2:00.0`, a kernel quirk, not a fleet device). The iGPU (`c5:00.0`) sits in group
|
||||
20 `identity` with ATS capability; interrupt remapping is enabled. frank-dragoon's
|
||||
boot ledger is unchanged to the MiB — 62.9 GiB weights (0.8 of them vision) + 5.5
|
||||
GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP
|
||||
depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all
|
||||
answered 200.
|
||||
|
||||
**The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
|
||||
Krackan/Strix Halo NPU rev 11), firmware `amdnpu/17f0_11/npu_7.sbin`, `Initialized
|
||||
amdxdna_accel_driver 0.7.0 … on minor 0`, IOMMU group 26 `identity`, PASID cap width
|
||||
10 (`Exec- Priv-`). `/dev/accel/accel0` is `crw-rw---- root:render 261,0` and `dank`
|
||||
is in `render`, so a quadlet needs `AddDevice=/dev/accel/accel0` +
|
||||
`GroupAdd=keep-groups`. **No NPU endpoint is deployed** — [[frank-npu]] stays the
|
||||
history of the retired endpoint; what changed is only its precondition.
|
||||
|
||||
**Deliberately deferred:** the `pp_dpm_fclk` pin (level 7, 2000 MHz) from the issue's
|
||||
co-run step. Pinning the fabric clock changes iGPU timing for every workload, and
|
||||
nothing co-runs on the NPU yet; it goes in when an NPU endpoint actually exists.
|
||||
PerfOpt still needs Linux 7.4 — `7.0.0-38-generic` has zero `perfopt` symbols and the
|
||||
26.04 archive caps at `7.0.0-38.38`.
|
||||
|
||||
**Tooling learned the hard way:** `/tmp` is tmpfs, so the reboot deleted the eval
|
||||
stage mid-A/B. `scripts/stage-frank-dragoon-eval.sh` rebuilds it byte-identically
|
||||
(corpus + probe PNG + battery), and must run through a login shell because `uv`
|
||||
lives in `~/.local/bin`.
|
||||
|
||||
## Related
|
||||
|
||||
- [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
|
||||
|
||||
Reference in new issue
Block a user