---
type: reference
title: frank-npu (retired) — FLM NPU endpoint on Dankclaw
description: The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the amd_iommu=off decision, and where every dependent route went.
tags:
  - flm
  - fastflowlm
  - xdna2
  - npu
  - dankclaw
  - frank-npu
  - retired
timestamp: 2026-09-11
---

# frank-npu — the retired FLM NPU endpoint

> **RETIRED 2026-09-11 by operator decision** — and still retired: there is no
> NPU endpoint service on Dankclaw today. The IOMMU configuration it required
> has since been reversed twice — see
> [[dankclaw-iommu-translation-off]]: the live 2026-10-18 posture is
> `cat /proc/cmdline` → `… ro quiet splash iommu=pt …` (no `amd_iommu` token),
> `amdxdna` is bound and `/dev/accel/accel0` exists, so the hardware is
> reachable again — revival is tracked as Gitea issue #3. The service unit, env
> file and serve script are staged in `serverAppConfigs/.AGENTTRASH/`
> (suffixed `.flm-npu`), and the host unit is still masked
> (`systemctl is-enabled frank-npu` → `masked`, read 2026-10-18). Port 7771
> belongs to [[frank-zealot-llama-prism-bonsai2-pq20]] today.

The third inference endpoint on Dankclaw: a FastFlowLM (FLM) v1.0.1 server
(`/usr/bin/flm`, dpkg `fastflowlm`) on the **XDNA2 NPU** (50 TOPS), serving the
whole local model registry on port 7771 and routing per request by the exact
FLM registry tag. Its one capability the iGPU fleet lacked was **sight** — and
briefly **sound**: the Gemma4 E4B IT NPU2 pack carries both a vision and an
audio tower, which is why the service ran that pack rather than the
uncensored-but-text-only alternative.

This page is kept as the retirement record. The operational detail below was
live-verified while the service ran and remains accurate about FLM's shape.

## Why it is retired

The replacement large-model runtime is worth **13–16 % of prefill** if the
kernel boots with `amd_iommu=off` instead of `iommu=pt`, measured by the
upstream project on this exact silicon. That single flag is the whole decision:

- With IOMMU off there is no PASID/SVA state, and the XDNA2 driver cannot hand
  out `/dev/accel/accel0` — every open fails with `err=-19` ("No such device").
  That is precisely the failure the endpoint was rescued from on 2026-08-30,
  recorded in the next section.
- `iommu=pt` was recorded here as "not a middle ground", on the theory that the
  passthrough domain still costs the vendor's prefill number. **That claim is
  dead: the host flipped to `iommu=pt` on 2026-10-18 and the local A/B on
  byte-identical traffic measured the cost at nothing** (+2.0 % cold prefill,
  1,616 t/s at 59k tokens). Identity mode does keep a 1:1 page walk plus
  ATS/PRI/PASID/GCR3 armed for the iGPU — worth 2–4 % on this silicon class, and
  **AMD PerfOpt** (Linux 7.4, merged 2026-09-24) removes it outright by enforcing
  only the IR/IW permission bits for the privileged iGPU — but nothing waits on
  that now.
- **The precondition is satisfied; the endpoint is still not deployed.** `amdxdna`
  binds `0000:c6:00.1` (driver 0.7.0, firmware `amdnpu/17f0_11/npu_7.sbin`), the
  NPU sits in IOMMU group 26 `identity` with PASID width 10, `/dev/accel/accel0`
  is `crw-rw---- root:render 261,0` and `dank` is in `render` — so a revival
  quadlet needs `AddDevice=/dev/accel/accel0` + `GroupAdd=keep-groups`, the
  ComfyUI `/dev/kfd` pattern. Re-read live 2026-10-18: `lspci -k -s c6:00.1` →
  `Kernel driver in use: amdxdna`; `journalctl -k -b` → `Initialized
  amdxdna_accel_driver 0.7.0 for 0000:c6:00.1` and `Load firmware
  amdnpu/17f0_11/npu_7.sbin`; `/sys/kernel/iommu_groups/26/type` → `identity`;
  `ls -l /dev/accel/accel0` → `crw-rw---- 1 root render 261, 0` (created
  2026-10-18 01:25); `id -nG dank` contains `render`. Kernel versions to avoid:
  **6.18–6.18.7** disables SVA outright *(unverified 2026-10-18 — upstream
  reports place the amdxdna/Strix-Halo breakage at 6.18.4+ and fixed by 6.19.2,
  not at a clean 6.18–6.18.7 window)*. Full census in
  [[dankclaw-iommu-translation-off]] § *2026-10-18 execution*.
- The operator took the vendor's prefill figure on the September evidence, so the
  NPU endpoint went — and it stays retired because nobody has asked for a model
  on it, not because the hardware is unreachable.

Consequences accepted at retirement: **audio input left the fleet with no
replacement** (halogen reads images only, no audio or video), and the surviving
image path is halogen's vision tower on `frank-dragoon`.

## Where its routes went

| consumer | setting | now |
|---|---|---|
| Hermes | `auxiliary.vision` | `dankclaw-large` / `frank-dragoon` (halogen vision tower) *(unverified 2026-10-18: `/opt/hermes` is gone from Dankclaw, so the live `config.yaml` could not be read)* |
| Hermes | `auxiliary.title_generation` | `dankserver-small` / `frank-zealot` *(unverified 2026-10-18: same)* |
| harness (`~/.omp/agent/config.yml`) | `modelRoles.vision` | moved again: `llamacpp (dankclaw-3090)/frank-zealot:xhigh` (`config.yml:6`) |
| harness | `modelRoles.tiny` | `llamacpp (dankclaw-3090)/frank-zealot:off` (`config.yml:10`) |
| harness | `modelRoles.task` | `llamacpp (dankclaw-3090)/frank-zealot:off` (`config.yml:11`) |

The `flm (dankclaw-npu)` provider block was removed from the harness model
config, and the `dankclaw-npu` provider from Hermes's live `config.yaml`.

## What it was

| Field | Value |
|---|---|
| Runtime | FLM v1.0.1 (`/usr/bin/flm`, dpkg `fastflowlm`; `dpkg -l fastflowlm` on 2026-10-18 still reports `fastflowlm 1.0.1`) on the XDNA2 NPU |
| Service | `frank-npu.service` (systemd **system** unit, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`) |
| Files (archived) | `scripts/frank-npu.env`, `scripts/serve-frank-npu.sh`, `scripts/frank-npu.service` |
| Port | `7771` on `0.0.0.0` (that port is frank-zealot's today) |
| Power mode | `FLM_PMODE=turbo` — nothing else used the NPU, so the top power mode cost no neighbour anything |
| Context | `FLM_CTX_LEN=131072` (128K, the verified maximum on this box) |
| Vision/audio | `FLM_ASR=0`; the E4B pack answers images through its own vision tower, and `FLM_ASR=1` would have added Whisper-V3-Turbo-NPU2 (+0.62 GB) and `/v1/audio/transcriptions` |

The serve script exported `FLM_CONFIG_PATH=/home/dank/.config/flm/model_list.json`
and `FLM_XCLBIN_PATH=/home/dank/.config/flm/xclbins`, with kernels symlinked
from `/opt/fastflowlm/share/flm/xclbins/` into the per-pack directories, so a
tag swap was an env-file edit rather than a script edit.

## API contract (as it behaved)

- The endpoint served the **whole local registry** (36 tags at first deploy) on
  one port; `/v1/models` listed them all. Chat completions routed per request
  by the exact registry tag in the `model` field — never a service alias.
- **Unknown tags did not 404**: `/v1/chat/completions` answered HTTP 200 with a
  flat error body `{"error":"Failed to load <tag> model!"}` and no `choices`.
  Status-code-only clients silently got an empty response.
- Health was `GET /v1/version`; FLM has **no `/health` route**.
- The NPU is a single-lane device — generations serialized FIFO. It was a
  background/triage endpoint, not interactive multi-user chat.

## The IOMMU/SVA root cause (first deploy crashed, fixed 2026-08-30)

First deploy crash-looped every 10 s with
`Error: Open /dev/accel/accel0 failed (err=-19): No such device`. The chain,
all live-verified on the host:

1. `amdxdna_drm_open` failed **SVA bind with ret -19** for every client — the
   XDNA2 driver cannot hand out the accel device without PASID/SVA.
2. SVA requires the AMD IOMMU to be enabled, and the kernel cmdline then
   carried `iommu=off`. With IOMMU off there are no IOMMU groups and no PASID
   state; `PASIDCtl: Enable-` was observed on `0000:c5:00.1`.
3. The fix was `iommu=pt`: passthrough keeps the iGPU default domain
   identity-mapped (no DMA-translation overhead for `frank-dragoon`) while
   enabling the IOMMU hardware and PASID that the driver's SVA bind needs.
4. Post-boot verification: `/sys/kernel/iommu_groups/` populated (30 groups),
   zero `SVA bind device failed` entries since boot, `accel0` recreated at
   boot, service active with `NRestarts=0`.

The same root cause explained the older
`amdgpu 0000:c4:00.0: no vmid pasid mapping supported` kernel line — a
consequence of IOMMU-off, not a GPU fault.

**The standing rule this produced — "never set `iommu=off` on this host again" —
was superseded twice**: `amd_iommu=off` was the intentional posture from
2026-10-16 to 2026-10-18, and the host has since flipped back to `iommu=pt`
(live `cat /proc/cmdline`, 2026-10-18: `… ro quiet splash iommu=pt …`). See
[[dankclaw-iommu-translation-off]] for the current posture and how to revert it.

## Client wiring (historical)

The pi/harness model config registered this endpoint with the **exact** FLM
registry tag as the model `id` and `baseUrl` `http://192.168.0.143:7771/v1`.
Both that provider block and Hermes's `dankclaw-npu` provider were removed at
retirement; the routes are mapped in the table above.

## Citations

- [FastFlowLM](https://github.com/FastFlowLM/FastFlowLM) — FLM runtime and NPU model packs
- [[dankclaw-iommu-translation-off]] — the decision that retired this endpoint
- [[halogen-flash-server]] — the surviving image-input path