docs(sightsound): NPU sight+sound runbook, fleet table, IOMMU/SVA root cause

frank-default-sightsound (FLM v1.0.1, XDNA2 NPU, port 52626) is live on
Dankclaw. Documents the first-deploy crash-loop root cause: iommu=off on the
kernel cmdline breaks the XDNA2 SVA bind (amdxdna_drm_open ret -19), so every
/dev/accel/accel0 open fails with ENODEV; fixed with iommu=pt + reboot.
Whole-registry single-port serving with per-request routing by exact FLM tag;
unknown tags fall back to llama3.2:1b (no 404). Health = GET /v1/version.
Vision + audio towers proven live (image_url color probe; audio control-pair
token-count probe). Updates AGENTS.md fleet table + runtime section, both
index pages, and the change log.
This commit is contained in:
Frank
2026-08-30 09:05:22 -07:00
parent 443adb04a7
commit 150412bbca
5 changed files with 184 additions and 1 deletions
+38 -1
View File
@@ -28,6 +28,7 @@ host.
|--------|---------|---------|-------|------|
| Dankserver | FreeToken v0.1.2 | frank-default-small | batsclamp Huihui-Qwen3.6-35B-A3B-abliterated-FP8 (block-fp8 e4m3 128x128, ~35 GB safetensors), experts host-RAM LRU, 262144 ctx / 4 concurrent, reasoning on | 7770 |
| Dankclaw | kingjones30/ROCmFPX llama.cpp (card-pinned fork) | frank-default-large | kingjones777 Qwen3.8-Flash-Next-ROCmFP4-STRIX (113.47 GiB single GGUF, Q4_0_ROCMFP4_STRIX, PLE streams via mmap), 401408 ctx / 2 slots, fa on, no spec decode, reasoning on | 7770 |
| Dankclaw | FastFlowLM (FLM) v1.0.1 (`flm serve`, dpkg `fastflowlm`) on the XDNA2 NPU | frank-default-sightsound | Gemma4-E4B-IT-NPU2 pack (vision + audio towers); whole local registry served on one port, per-request routing by exact FLM tag; `iommu=pt` boot arg mandatory (SVA bind needs enabled IOMMU) | 52626 |
History: the 2026-08-17 redo retired the pipeline-parallel split and the
FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
@@ -45,7 +46,9 @@ Dankserver moved to the FreeToken runtime serving the official
Qwen3.6-35B-A3B-FP8. On 2026-08-30 both services swapped again:
Dankclaw to the kingjones STRIX model on the card's pinned
kingjones30/ROCmFPX runtime, Dankserver to the Huihui abliterated
FP8 checkpoint on FreeToken. Runbooks:
FP8 checkpoint on FreeToken. On 2026-08-30 the NPU endpoint returned as
`frank-default-sightsound` — FLM v1.0.1 on XDNA2 serving sight+sound (see the
Runtime & Troubleshooting section and its runbook). Runbooks:
[serverAppConfigs-docs/LLM/index.md](serverAppConfigs-docs/LLM/index.md).
Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`.
@@ -279,6 +282,40 @@ When you rename a service or move a model to a new host, update
## Runtime & Troubleshooting
### frank-default-sightsound — FLM NPU sight+sound endpoint (Dankclaw, ACTIVE)
The third inference endpoint on Dankclaw. The `flm serve` runtime (dpkg
package `fastflowlm`, v1.0.1, binary `/usr/bin/flm`) serves the **whole local
model registry** (36 tags at first deploy) on a single port — 52626 — and
routes chat completions per request by the exact FLM registry tag in the
`model` field. Unknown tags do NOT 404: they silently serve `llama3.2:1b`,
so clients must send exact registry tags, never service aliases. Primary
model = Gemma4 E4B IT NPU2 pack (vision + audio towers); the uncensored
candidate (`qwythos:9b`) is text-only and fails the modality requirement, so
this endpoint trades "uncensored" for "sight + sound". The NPU is a
single-lane device — generations serialize FIFO; treat this as a
background/triage endpoint, not an interactive multi-user chat.
Hard requirements verified at first deploy (2026-08-30):
- **IOMMU must be enabled.** With `iommu=off` on the kernel cmdline every
open of `/dev/accel/accel0` fails with `err=-19`: the XDNA2 driver's SVA
bind needs PASID/SVA, which requires enabled IOMMU hardware. Fix applied:
`iommu=pt` in `GRUB_CMDLINE_LINUX_DEFAULT` + reboot (passthrough keeps the
iGPU default domain identity-mapped — zero DMA-translation overhead for
frank-default-large — while enabling IOMMU + PASID; PASID domains are
paged regardless of the default-domain type). Never set `iommu=off` on
this host again; it silently kills the NPU.
- Health probe = `GET /v1/version` (FLM has no `/health` route).
- Unit: `User=dank`, `LimitMEMLOCK=infinity` (FLM requires unlimited
memlock), `Restart=on-failure`, `RestartSec=10`. Deploy via
`scripts/deploy.sh dankclaw --service frank-default-sightsound`.
- Verified live at first deploy: image probe (red PNG as `image_url`
data-URI → correct color answer, vision tower consumed the pixels) and an
audio control-pair probe (tone WAV via `input_audio` consumed ~22
audio-tower tokens vs a text-only control).
Runbook: [serverAppConfigs-docs/LLM/frank-default-sightsound.md](serverAppConfigs-docs/LLM/frank-default-sightsound.md).
### kingjones Qwen3.8-Flash-Next STRIX on kingjones30/ROCmFPX (Dankclaw, ACTIVE)
The large service serves