docs(iommu): record the executed flip to iommu=pt and its measured cost of nothing
Window 01:24-01:35. Byte-identical A/B (rebuild proved it: same 1,287,656 B corpus, same billed token counts 59,024 / 21,568): cold prefill 1,585.1 -> 1,616.0 t/s (+2.0 %), warm 208,531 -> 214,131, needle hit/hit both boots. The 13-16 % that justified amd_iommu=off since 2026-09-11 does not exist on this host; the decode delta (-9.7 %) is instrument noise -- three repeats of one request on the same boot gave 31.8/46.9/47.8 t/s. NPU precondition restored: amdxdna bound 0000:c6:00.1 (driver 0.7.0, firmware amdnpu/17f0_11/npu_7.sbin), group 26 identity, PASID width 10, /dev/accel/accel0 root:render with dank in render. No endpoint deployed, so frank-npu stays retired -- only its rationale changes. Gates: 0 failed units, 32 groups (31 identity + the SD reader's DMA quirk), iGPU c5:00.0 identity with ATS, dragoon ledger unchanged (74.1 GiB, no clamp, listening at 39.8 s). Deferred on purpose: the pp_dpm_fclk pin (changes iGPU timing for everyone, nothing co-runs yet) and PerfOpt (7.0.0-38 has zero perfopt symbols, 26.04 caps at 7.0.0-38.38). Re-keyed: decision record banner/front matter/verification/revert + new execution section, frank-npu rationale, AGENTS key lesson, both index pages.
This commit is contained in:
Frank
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: research
|
||||
title: Dankclaw IOMMU translation off (amd_iommu=off) — decision record
|
||||
description: Why Dankclaw boots with amd_iommu=off since 2026-09-11, what it buys (a vendor-claimed 13-16% prefill for frank-dragoon, never measured locally), what it costs (the XDNA2 NPU endpoint, machine-wide DMA translation), the 2026-10-18 AMD PerfOpt research that puts the trade on an expiring path, and how to revert.
|
||||
title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)
|
||||
description: Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path.
|
||||
tags:
|
||||
- dankclaw
|
||||
- iommu
|
||||
@@ -16,10 +16,11 @@ timestamp: 2026-09-11
|
||||
|
||||
# Dankclaw IOMMU translation off — decision record
|
||||
|
||||
**Decision (2026-09-11):** Dankclaw boots with **`amd_iommu=off`** instead of
|
||||
`iommu=pt`, to recover the prefill throughput the large-model runtime loses to
|
||||
IOMMU translation. The XDNA2 NPU endpoint was accepted as the cost and is
|
||||
retired ([[frank-npu]]).
|
||||
**Current state (since 2026-10-18 01:24): Dankclaw boots `iommu=pt`** — identity
|
||||
DMA, IOMMU armed, XDNA2 NPU bound and usable. Everything below is the record of
|
||||
the 2026-09-11 decision that booted it with **`amd_iommu=off`** on a vendor-claimed
|
||||
13–16 % prefill figure, what that cost, and the measured flip back: see
|
||||
§ *2026-10-18 execution* at the bottom for the numbers and the revert path.
|
||||
|
||||
## What changed
|
||||
|
||||
@@ -77,23 +78,26 @@ so only that one was taken:
|
||||
## Verification and revert
|
||||
|
||||
```bash
|
||||
cat /proc/cmdline # expect amd_iommu=off, no iommu=pt
|
||||
ls /sys/kernel/iommu_groups/ | wc -l # expect 0 groups (was 30)
|
||||
systemctl is-active frank-npu # expect inactive (masked)
|
||||
cat /proc/cmdline # expect iommu=pt, no amd_iommu=off
|
||||
ls /sys/kernel/iommu_groups/ | wc -l # expect 32 (0 means the flip is gone)
|
||||
cat /sys/kernel/iommu_groups/*/type | sort | uniq -c # 31 identity + 1 DMA (the SD reader)
|
||||
sudo dmesg | grep -m1 amdxdna # driver bound + firmware loaded = NPU alive
|
||||
curl -s http://127.0.0.1:7770/health # frank-dragoon must still answer 200
|
||||
```
|
||||
|
||||
**To revert:** put `iommu=pt` back in `GRUB_CMDLINE_LINUX_DEFAULT` in place of
|
||||
`amd_iommu=off`, run `sudo update-grub`, reboot, and unmask/start `frank-npu`
|
||||
with the archived unit (`.AGENTTRASH/frank-npu.service.flm-npu`) if the NPU
|
||||
endpoint is wanted back. The IOMMU change is independent of the halogen
|
||||
deployment — reverting one does not require reverting the other.
|
||||
**To revert:** set `GRUB_CMDLINE_LINUX_DEFAULT="quiet splash amd_iommu=off"` in
|
||||
the tracked `dankclaw/host/grub` and run `./scripts/set-dankclaw-bootline.sh
|
||||
--reboot` on Dankclaw. Never hand-edit `/etc/default/grub`: the tracked file is
|
||||
the source, and the installer diffs it, installs it 0644 root:root, runs
|
||||
`update-grub`, and checks the default `grub.cfg` entry before it will reboot. The
|
||||
IOMMU change is independent of the halogen deployment — reverting one does not
|
||||
require reverting the other.
|
||||
|
||||
## 2026-10-18 research: the penalty now has a hardware fix (AMD PerfOpt)
|
||||
|
||||
Asked and answered — **the decision changes, the host does not.** Nothing was
|
||||
changed on Dankclaw today; this section records why the figure behind the
|
||||
decision is on an expiring path — and that it was never measured on this host.
|
||||
Asked and answered the same day the flip was executed (§ *2026-10-18 execution*
|
||||
below): the figure behind the September decision is on an expiring path, and — the
|
||||
part that actually mattered — it was **never measured on this host**.
|
||||
|
||||
**Mechanism.** AMD IOMMU **PerfOpt**, spec-defined in the AMD I/O Virtualization
|
||||
Technology (IOMMU) spec §3.4.9: MMIO `0x016C` bit 13 `PERF_OPT_EN`, gated by
|
||||
@@ -159,6 +163,65 @@ still an explicit OK per run):
|
||||
domain. Confirm with `dmesg | grep -i perfopt`; force off with
|
||||
`amdgpu.iommu_perfopt=0` while the knob still exists.
|
||||
|
||||
## 2026-10-18 execution: the flip, measured on this host
|
||||
|
||||
Executed 01:24–01:35 with Frank's go-ahead. `GRUB_CMDLINE_LINUX_DEFAULT` went from
|
||||
`quiet splash amd_iommu=off` to `quiet splash iommu=pt` through the now-tracked
|
||||
`dankclaw/host/grub` + `scripts/set-dankclaw-bootline.sh` (diff → install 0644
|
||||
root:root → `update-grub` → verify the default `grub.cfg` entry → reboot only on
|
||||
`--reboot`).
|
||||
|
||||
**The A/B, on byte-identical traffic.** The corpus is the rebuilt wikitext-2 raw
|
||||
test split — 2,891 lines, **1,287,656 B**, the same file the pre-reboot boot read,
|
||||
and the engine confirms it by billing the same tokens (59,024 prefill prompt,
|
||||
21,568 needle haystack). Both arms ran `battery-frank-dragoon.py … tail` against
|
||||
the live service with nothing else on the iGPU; the pt arm pre-warmed the 51.2 GB
|
||||
n-gram table with a full read (40 s) so a cold page cache could not masquerade as
|
||||
IOMMU cost.
|
||||
|
||||
| probe | `amd_iommu=off` | `iommu=pt` |
|
||||
|---|---|---|
|
||||
| cold prefill, 59,024 tokens | 1,585.1 t/s | **1,616.0 t/s (+2.0 %)** |
|
||||
| warm prefill (prompt cache) | 208,531 t/s | 214,131 t/s |
|
||||
| cache ratio | 131.6× | 132.5× |
|
||||
| needle hi/lo @ 21,568 tokens | hit / hit | hit / hit |
|
||||
| decode, 512 sampled tokens | 37.3 t/s | 33.7 t/s |
|
||||
|
||||
**Prefill cost: none.** Identity mode is not slower here — it is +2.0 %, inside
|
||||
run-to-run spread. The single decode sample moved −9.7 %, and that instrument
|
||||
cannot see anything: three immediate repeats on the same boot gave 31.8, 46.9 and
|
||||
47.8 t/s on identical requests, so any single-sample decode delta is noise. The
|
||||
13–16 % that justified `amd_iommu=off` for five weeks does not exist on this host,
|
||||
and PerfOpt is now a bonus nobody is waiting for rather than the reason to flip.
|
||||
|
||||
**Post-boot gates.** 0 failed units, `systemd` running. 32 IOMMU groups — 31
|
||||
`identity` + 1 `DMA`, and that one belongs to the Genesys Logic SD card reader
|
||||
(`c2:00.0`, a kernel quirk, not a fleet device). The iGPU (`c5:00.0`) sits in group
|
||||
20 `identity` with ATS capability; interrupt remapping is enabled. frank-dragoon's
|
||||
boot ledger is unchanged to the MiB — 62.9 GiB weights (0.8 of them vision) + 5.5
|
||||
GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP
|
||||
depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all
|
||||
answered 200.
|
||||
|
||||
**The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
|
||||
Krackan/Strix Halo NPU rev 11), firmware `amdnpu/17f0_11/npu_7.sbin`, `Initialized
|
||||
amdxdna_accel_driver 0.7.0 … on minor 0`, IOMMU group 26 `identity`, PASID cap width
|
||||
10 (`Exec- Priv-`). `/dev/accel/accel0` is `crw-rw---- root:render 261,0` and `dank`
|
||||
is in `render`, so a quadlet needs `AddDevice=/dev/accel/accel0` +
|
||||
`GroupAdd=keep-groups`. **No NPU endpoint is deployed** — [[frank-npu]] stays the
|
||||
history of the retired endpoint; what changed is only its precondition.
|
||||
|
||||
**Deliberately deferred:** the `pp_dpm_fclk` pin (level 7, 2000 MHz) from the issue's
|
||||
co-run step. Pinning the fabric clock changes iGPU timing for every workload, and
|
||||
nothing co-runs on the NPU yet; it goes in when an NPU endpoint actually exists.
|
||||
PerfOpt still needs Linux 7.4 — `7.0.0-38-generic` has zero `perfopt` symbols and the
|
||||
26.04 archive caps at `7.0.0-38.38`.
|
||||
|
||||
**Tooling learned the hard way:** `/tmp` is tmpfs, so the reboot deleted the eval
|
||||
stage mid-A/B. `scripts/stage-frank-dragoon-eval.sh` rebuilds it byte-identically
|
||||
(corpus + probe PNG + battery), and must run through a login shell because `uv`
|
||||
lives in `~/.local/bin`.
|
||||
|
||||
## Related
|
||||
|
||||
- [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
|
||||
|
||||
@@ -43,20 +43,25 @@ upstream project on this exact silicon. That single flag is the whole decision:
|
||||
That is precisely the failure the endpoint was rescued from on 2026-08-30,
|
||||
recorded in the next section.
|
||||
- `iommu=pt` was recorded here as "not a middle ground", on the theory that the
|
||||
passthrough domain still costs the vendor's prefill number. **That claim did
|
||||
not survive the 2026-10-18 research.** Identity mode is not free for the iGPU
|
||||
— its DMA still walks 1:1 page tables and the AMD attach path enables
|
||||
ATS/PRI/PASID/GCR3 for the device — but on this silicon class the cost is
|
||||
single-digit percent, and **AMD PerfOpt** (Linux 7.4, merged 2026-09-24)
|
||||
removes it: the IOMMU then enforces only the IR/IW permission bits for the
|
||||
privileged iGPU and stays fully armed for every other device, the NPU
|
||||
included. So `pt` IS the revival path for this endpoint. See
|
||||
[[dankclaw-iommu-translation-off]] § *2026-10-18 research* for the kernel
|
||||
census, the versions to avoid (6.18–6.18.7 disable SVA), the device-access
|
||||
quadlet law (`AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups`), and the
|
||||
measurement step that must precede any decision.
|
||||
passthrough domain still costs the vendor's prefill number. **That claim is
|
||||
dead: the host flipped to `iommu=pt` on 2026-10-18 and the local A/B on
|
||||
byte-identical traffic measured the cost at nothing** (+2.0 % cold prefill,
|
||||
1,616 t/s at 59k tokens). Identity mode does keep a 1:1 page walk plus
|
||||
ATS/PRI/PASID/GCR3 armed for the iGPU — worth 2–4 % on this silicon class, and
|
||||
**AMD PerfOpt** (Linux 7.4, merged 2026-09-24) removes it outright by enforcing
|
||||
only the IR/IW permission bits for the privileged iGPU — but nothing waits on
|
||||
that now.
|
||||
- **The precondition is satisfied; the endpoint is still not deployed.** `amdxdna`
|
||||
binds `0000:c6:00.1` (driver 0.7.0, firmware `amdnpu/17f0_11/npu_7.sbin`), the
|
||||
NPU sits in IOMMU group 26 `identity` with PASID width 10, `/dev/accel/accel0`
|
||||
is `crw-rw---- root:render 261,0` and `dank` is in `render` — so a revival
|
||||
quadlet needs `AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups`, the
|
||||
ComfyUI `/dev/kfd` pattern. Kernel versions to avoid: **6.18–6.18.7** disables
|
||||
SVA outright. Full census in [[dankclaw-iommu-translation-off]]
|
||||
§ *2026-10-18 execution*.
|
||||
- The operator took the vendor's prefill figure on the September evidence, so the
|
||||
NPU endpoint went.
|
||||
NPU endpoint went — and it stays retired because nobody has asked for a model
|
||||
on it, not because the hardware is unreachable.
|
||||
|
||||
Consequences accepted at retirement: **audio input left the fleet with no
|
||||
replacement** (halogen reads images only, no audio or video), and the surviving
|
||||
|
||||
@@ -463,10 +463,12 @@ frank-zealot, port 7771]
|
||||
mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090
|
||||
(dense/attention/KV) + LPDDR (experts/PLE) hybrid; its measured envelope and
|
||||
decode root-cause chain are the record of why the fleet went back to halogen
|
||||
- [[dankclaw-iommu-translation-off]] — why Dankclaw boots with
|
||||
`amd_iommu=off` since 2026-09-11 (13-16 % prefill, measured upstream),
|
||||
what it cost (the NPU endpoint), what was deliberately not copied from
|
||||
the reference machine, and how to revert
|
||||
- [[dankclaw-iommu-translation-off]] — Dankclaw's IOMMU boot line: **`iommu=pt`**
|
||||
since 2026-10-18, measured on this host at no prefill cost (+2.0 %, 1,616 t/s
|
||||
cold), which restored the XDNA2 NPU; the `amd_iommu=off` era (2026-09-11) it
|
||||
replaced ran on a 13-16 % figure measured upstream only; includes the AMD
|
||||
PerfOpt research, what was deliberately not copied from the reference machine,
|
||||
and the revert path via `scripts/set-dankclaw-bootline.sh`
|
||||
- [[kingjones-strix-rocmfp4]] — PRIOR frank-dragoon runbook and rollback
|
||||
target: kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF, 177B params
|
||||
(56B MoE + 51.2B PLE), single 113.47 GiB GGUF, on the card's pinned
|
||||
|
||||
@@ -387,9 +387,9 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
and rollback target (2026-09-15 → 09-16): ik_llama.cpp serving
|
||||
mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090 +
|
||||
Strix Halo hybrid — its measured envelope is why the fleet went back
|
||||
- [[dankclaw-iommu-translation-off]] — why Dankclaw boots with
|
||||
`amd_iommu=off` since 2026-09-11 (13-16 % prefill), what it cost
|
||||
(the NPU endpoint), and how to revert
|
||||
- [[dankclaw-iommu-translation-off]] — Dankclaw's IOMMU boot line: `iommu=pt`
|
||||
since 2026-10-18 (measured cost of identity DMA: nothing — +2.0 % prefill), the
|
||||
`amd_iommu=off` era it replaced, and the tracked file + installer that own it
|
||||
- [[kingjones-strix-rocmfp4]] — the PRIOR frank-dragoon runbook
|
||||
(2026-08-30 → 2026-09-11) and the rollback target:
|
||||
kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF on the card's
|
||||
|
||||
@@ -2066,3 +2066,45 @@ every restart since the cutover.
|
||||
vendor number, and the plan posted to Gitea issue #3 (comment 23, then edited).
|
||||
bats 371/371; commits `7643381` plus this correction. Nothing deployed — no
|
||||
service was touched.
|
||||
|
||||
## 2026-10-18 — boot line flipped: Dankclaw now boots `iommu=pt`
|
||||
|
||||
- Window executed 01:24–01:35 on Frank's go-ahead, with the A/B the research
|
||||
demanded. Provenance first: `/etc/default/grub` had never been in the repo
|
||||
while the IOMMU line was the most consequential kernel decision on the box, so
|
||||
it is now tracked at `dankclaw/host/grub` (fetched byte-exact, 1581 B) and moves
|
||||
only through `scripts/set-dankclaw-bootline.sh` — diff, install 0644 root:root,
|
||||
`update-grub`, **verify the default `grub.cfg` entry**, reboot on `--reboot`
|
||||
only. The deployer plans nothing under `dankclaw/host/`: kernel state is not a
|
||||
container restart.
|
||||
- **Measured arms, byte-identical traffic.** OFF arm at uptime 2 d 23 h: cold
|
||||
prefill **1,585.1 t/s** (59,024 tok), warm 208,531, cache ratio 131.6×, needle
|
||||
hit/hit @21,568, decode 37.3 t/s. PT arm after the reboot: **1,616.0 t/s
|
||||
(+2.0 %)**, warm 214,131, ratio 132.5×, needle hit/hit, decode 33.7 t/s. The
|
||||
corpus was rebuilt after the reboot deleted `/tmp` (tmpfs) and is provably the
|
||||
same input: 2,891 lines, **1,287,656 B**, and the engine billed identical token
|
||||
counts. Verdict: **identity DMA costs nothing on this host** — the 13–16 % that
|
||||
justified `amd_iommu=off` for five weeks never existed here. Decode's −9.7 % is
|
||||
instrument noise: three repeats of the same request on the same boot gave 31.8,
|
||||
46.9, 47.8 t/s.
|
||||
- Gates after boot: 0 failed units, systemd running, 32 IOMMU groups (31 identity
|
||||
+ 1 DMA = the Genesys Logic SD reader `c2:00.0`, a kernel quirk), iGPU `c5:00.0`
|
||||
identity group 20 with ATS, interrupt remapping on. frank-dragoon's ledger is
|
||||
byte-for-byte the same shape (62.9 + 5.5 + 5.7 = 74.1 GiB, 2 slots over ONE
|
||||
200,704 pool, MTP depth 2, prompt cache ON, no clamp; listening at 39.8 s), and
|
||||
Gitea / Caddy edge / Grafana answered 200.
|
||||
- **The XDNA2 NPU is back:** `amdxdna` bound `0000:c6:00.1`, firmware
|
||||
`amdnpu/17f0_11/npu_7.sbin`, `amdxdna_accel_driver 0.7.0 … minor 0`, group 26
|
||||
`identity`, PASID width 10. `/dev/accel/accel0` `crw-rw---- root:render 261,0`
|
||||
with `dank` in `render`. No endpoint deployed — revival needs a quadlet with
|
||||
`AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups`.
|
||||
- Deliberately NOT done: the `pp_dpm_fclk` level-7 pin (it changes iGPU timing for
|
||||
every workload and nothing co-runs on the NPU yet) and any PerfOpt work (kernel
|
||||
`7.0.0-38` has zero `perfopt` symbols; the 26.04 archive caps at `7.0.0-38.38`).
|
||||
- New tool: `scripts/stage-frank-dragoon-eval.sh` rebuilds `/tmp/halogen-eval`
|
||||
byte-identically (corpus + probe PNG + battery) — run it through `bash -lc`,
|
||||
`uv` is in `~/.local/bin`. Docs re-keyed: decision record (banner, front matter,
|
||||
verification/revert, new § *2026-10-18 execution*), `frank-npu.md` (precondition
|
||||
satisfied, endpoint still absent), AGENTS.md key lesson, both index pages.
|
||||
bats 371/371. frank-dragoon and frank-zealot were live the whole window; only
|
||||
the reboot interrupted them (~40 s to listen, health 200).
|
||||
|
||||
Reference in New Issue
Block a user