docs(sightsound): NPU sight+sound runbook, fleet table, IOMMU/SVA root cause
frank-default-sightsound (FLM v1.0.1, XDNA2 NPU, port 52626) is live on Dankclaw. Documents the first-deploy crash-loop root cause: iommu=off on the kernel cmdline breaks the XDNA2 SVA bind (amdxdna_drm_open ret -19), so every /dev/accel/accel0 open fails with ENODEV; fixed with iommu=pt + reboot. Whole-registry single-port serving with per-request routing by exact FLM tag; unknown tags fall back to llama3.2:1b (no 404). Health = GET /v1/version. Vision + audio towers proven live (image_url color probe; audio control-pair token-count probe). Updates AGENTS.md fleet table + runtime section, both index pages, and the change log.
This commit is contained in:
@@ -28,6 +28,7 @@ host.
|
||||
|--------|---------|---------|-------|------|
|
||||
| Dankserver | FreeToken v0.1.2 | frank-default-small | batsclamp Huihui-Qwen3.6-35B-A3B-abliterated-FP8 (block-fp8 e4m3 128x128, ~35 GB safetensors), experts host-RAM LRU, 262144 ctx / 4 concurrent, reasoning on | 7770 |
|
||||
| Dankclaw | kingjones30/ROCmFPX llama.cpp (card-pinned fork) | frank-default-large | kingjones777 Qwen3.8-Flash-Next-ROCmFP4-STRIX (113.47 GiB single GGUF, Q4_0_ROCMFP4_STRIX, PLE streams via mmap), 401408 ctx / 2 slots, fa on, no spec decode, reasoning on | 7770 |
|
||||
| Dankclaw | FastFlowLM (FLM) v1.0.1 (`flm serve`, dpkg `fastflowlm`) on the XDNA2 NPU | frank-default-sightsound | Gemma4-E4B-IT-NPU2 pack (vision + audio towers); whole local registry served on one port, per-request routing by exact FLM tag; `iommu=pt` boot arg mandatory (SVA bind needs enabled IOMMU) | 52626 |
|
||||
|
||||
History: the 2026-08-17 redo retired the pipeline-parallel split and the
|
||||
FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
|
||||
@@ -45,7 +46,9 @@ Dankserver moved to the FreeToken runtime serving the official
|
||||
Qwen3.6-35B-A3B-FP8. On 2026-08-30 both services swapped again:
|
||||
Dankclaw to the kingjones STRIX model on the card's pinned
|
||||
kingjones30/ROCmFPX runtime, Dankserver to the Huihui abliterated
|
||||
FP8 checkpoint on FreeToken. Runbooks:
|
||||
FP8 checkpoint on FreeToken. On 2026-08-30 the NPU endpoint returned as
|
||||
`frank-default-sightsound` — FLM v1.0.1 on XDNA2 serving sight+sound (see the
|
||||
Runtime & Troubleshooting section and its runbook). Runbooks:
|
||||
[serverAppConfigs-docs/LLM/index.md](serverAppConfigs-docs/LLM/index.md).
|
||||
Retired scripts live in `old/` (reference only) and `.AGENTTRASH/`.
|
||||
|
||||
@@ -279,6 +282,40 @@ When you rename a service or move a model to a new host, update
|
||||
|
||||
## Runtime & Troubleshooting
|
||||
|
||||
### frank-default-sightsound — FLM NPU sight+sound endpoint (Dankclaw, ACTIVE)
|
||||
|
||||
The third inference endpoint on Dankclaw. The `flm serve` runtime (dpkg
|
||||
package `fastflowlm`, v1.0.1, binary `/usr/bin/flm`) serves the **whole local
|
||||
model registry** (36 tags at first deploy) on a single port — 52626 — and
|
||||
routes chat completions per request by the exact FLM registry tag in the
|
||||
`model` field. Unknown tags do NOT 404: they silently serve `llama3.2:1b`,
|
||||
so clients must send exact registry tags, never service aliases. Primary
|
||||
model = Gemma4 E4B IT NPU2 pack (vision + audio towers); the uncensored
|
||||
candidate (`qwythos:9b`) is text-only and fails the modality requirement, so
|
||||
this endpoint trades "uncensored" for "sight + sound". The NPU is a
|
||||
single-lane device — generations serialize FIFO; treat this as a
|
||||
background/triage endpoint, not an interactive multi-user chat.
|
||||
|
||||
Hard requirements verified at first deploy (2026-08-30):
|
||||
|
||||
- **IOMMU must be enabled.** With `iommu=off` on the kernel cmdline every
|
||||
open of `/dev/accel/accel0` fails with `err=-19`: the XDNA2 driver's SVA
|
||||
bind needs PASID/SVA, which requires enabled IOMMU hardware. Fix applied:
|
||||
`iommu=pt` in `GRUB_CMDLINE_LINUX_DEFAULT` + reboot (passthrough keeps the
|
||||
iGPU default domain identity-mapped — zero DMA-translation overhead for
|
||||
frank-default-large — while enabling IOMMU + PASID; PASID domains are
|
||||
paged regardless of the default-domain type). Never set `iommu=off` on
|
||||
this host again; it silently kills the NPU.
|
||||
- Health probe = `GET /v1/version` (FLM has no `/health` route).
|
||||
- Unit: `User=dank`, `LimitMEMLOCK=infinity` (FLM requires unlimited
|
||||
memlock), `Restart=on-failure`, `RestartSec=10`. Deploy via
|
||||
`scripts/deploy.sh dankclaw --service frank-default-sightsound`.
|
||||
- Verified live at first deploy: image probe (red PNG as `image_url`
|
||||
data-URI → correct color answer, vision tower consumed the pixels) and an
|
||||
audio control-pair probe (tone WAV via `input_audio` consumed ~22
|
||||
audio-tower tokens vs a text-only control).
|
||||
|
||||
Runbook: [serverAppConfigs-docs/LLM/frank-default-sightsound.md](serverAppConfigs-docs/LLM/frank-default-sightsound.md).
|
||||
### kingjones Qwen3.8-Flash-Next STRIX on kingjones30/ROCmFPX (Dankclaw, ACTIVE)
|
||||
|
||||
The large service serves
|
||||
|
||||
Reference in New Issue
Block a user