Files

T

Frank 55f87fcdce llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot

The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.

2026-10-10 17:03:14 -07:00

4.3 KiB

Raw Blame History

type, title, description, tags, timestamp

type

title

description

Dankclaw IOMMU translation off — decision record

Decision (2026-09-11): Dankclaw boots with amd_iommu=off instead of iommu=pt, to recover the prefill throughput the large-model runtime loses to IOMMU translation. The XDNA2 NPU endpoint was accepted as the cost and is retired (frank-npu).

What changed

GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub on Dankclaw:

# before
ro quiet splash iommu=pt crashkernel=…

# after
ro quiet splash amd_iommu=off crashkernel=…

Then sudo update-grub and a reboot. Nothing else on the command line was touched.

Evidence

The upstream project published the measurement, and it is the only A/B they ran: 460 → 385 tok/s prefill at a 2,048-token prompt, 13 to 16 percent, taken on a Ryzen AI Max+ 395 with 128 GB — the same SKU as Dankclaw. Their published serving profile boots this way, so every number in their README is a number with translation off. Decode is unaffected; prefill is where the loss is.

The mechanism offered is DMA translation on the path every GPU allocation takes. The project states plainly that this is their configuration and not a tuning guide, that the flag turns off DMA translation machine-wide, and that it takes the NPU with it.

What it costs

The NPU endpoint dies. With IOMMU off there is no PASID/SVA state and /dev/accel/accel0 cannot be opened — err=-19, the exact failure documented in frank-npu. Audio input left the fleet with it; image input survives on frank-dragoon through halogen's vision tower.
Posture change machine-wide. Dankclaw also serves Caddy (the TLS edge and the Anubis/Authentik guard chain), Authentik, Pi-hole, Gitea, and the observability stack. DMA translation is now off for all of them. This was accepted explicitly as the price of the prefill figure.

What was deliberately NOT copied

The reference machine's command line has six settings. Only one was measured, so only that one was taken:

setting	reference value	why not here
`amd_iommu=off`	set	taken — the measured 13–16 %
`amdgpu.gttsize`	`126976` MiB	a size, not a constant: it must equal installed RAM (their table: 128 GB → 126976, 96 GB → 95232, 64 GB → 63488). Dankclaw's GTT already reports 114 GiB ≈ 94 % of its 121 GiB without the flag, so it would buy nothing
`ttm.pages_limit`	`32505856`	the same number in 4 KiB pages (their 128 GB row is 99.3 % of RAM). Pasting it onto a different machine asks the driver for more GTT than exists
`amdgpu.vm_update_mode=0`	set	upstream has never run without it, makes no claim about what it buys, and lists it "for completeness rather than recommended"
`amdgpu.noretry=0`	set	as above
`amdgpu.sg_display=0`	set	as above

Verification and revert

cat /proc/cmdline                        # expect amd_iommu=off, no iommu=pt
ls /sys/kernel/iommu_groups/ | wc -l     # expect 0 groups (was 30)
systemctl is-active frank-npu            # expect inactive (masked)
curl -s http://127.0.0.1:7770/health     # frank-dragoon must still answer 200

To revert: put iommu=pt back in GRUB_CMDLINE_LINUX_DEFAULT in place of amd_iommu=off, run sudo update-grub, reboot, and unmask/start frank-npu with the archived unit (.AGENTTRASH/frank-npu.service.flm-npu) if the NPU endpoint is wanted back. The IOMMU change is independent of the halogen deployment — reverting one does not require reverting the other.

frank-npu — the endpoint this retired, including the err=-19 root cause
halogen-flash-server — the runtime bought by the prefill figure
kingjones-strix-rocmfp4 — the runtime it replaced (rollback target)

Citations

peonist-ai/halogen-flash-server — README "The host settings these numbers were measured on", CHANGELOG 0.5.5–0.5.8

4.3 KiB Raw Blame History Unescape Escape