dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

sightsound: move FLM NPU endpoint port 52626 -> 7771

Browse Source 
Env file is the source of truth (unit ExecStart runs the serve script which
sources it); serve-script default and the deployer health-gate URL flipped in
lockstep so the post-restart health gate polls the new port. Docs, fleet table,
and both index bullets updated to match.
This commit is contained in:
Frank
2026-08-30 14:06:54 -07:00
parent 9b9f0f9051
commit 9ef034e5e6
7 changed files with 10 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/LLM/frank-default-sightsound.md
+3 -3
View File
@@ -1,7 +1,7 @@
---
type: configuration
title: frank-default-sightsound Runbook
description: FastFlowLM (FLM) NPU sight+sound endpoint on Dankclaw — XDNA2 NPU serving the Gemma4 E4B vision+audio pack on port 52626, including the IOMMU/SVA root cause that took it down at first deploy.
description: FastFlowLM (FLM) NPU sight+sound endpoint on Dankclaw — XDNA2 NPU serving the Gemma4 E4B vision+audio pack on port 7771, including the IOMMU/SVA root cause that took it down at first deploy.
tags:
- flm
- fastflowlm
@@ -32,7 +32,7 @@ is text-only and fails the hard modality requirement, so this endpoint trades
| Service | `frank-default-sightsound.service` (systemd system unit, `User=dank`, `Restart=on-failure`, `RestartSec=10`, `LimitMEMLOCK=infinity`) |
| Files | `scripts/frank-default-sightsound.env`, `scripts/serve-frank-default-sightsound.sh`, `scripts/frank-default-sightsound.service` |
| Deploy | `scripts/deploy.sh dankclaw --service frank-default-sightsound` (relink + daemon-reload + enable + restart; health gate polls `/v1/version`, 300 s cap — FLM has **no `/health` route**) |
| Port | `52626` on `0.0.0.0` (52625 left free for ad-hoc `flm serve` experiments) |
| Port | `7771` on `0.0.0.0` (52625/52626 left free for ad-hoc `flm serve` experiments) |
| Power mode | `FLM_PMODE=turbo` — no other tenant uses the NPU on this host (the large model runs on the iGPU), so the top power mode costs no neighbour anything; drop to `performance` if iGPU power-budget contention ever shows |
| Context | `FLM_CTX_LEN` = verified max for the E4B pack on this box (native 128K context; the registry's default context length is only the fallback). FLM rounds non-power-of-2 values up. |
| ASR | `FLM_ASR=0` — the E4B pack answers audio natively through its own audio tower in `/v1/chat/completions`; setting `FLM_ASR=1` additionally loads Whisper-V3-Turbo-NPU2 (+0.62 GB) and exposes `/v1/audio/transcriptions` |
@@ -106,6 +106,6 @@ fault. Do not set `iommu=off` on this host again; it silently kills the NPU.
The pi client model config (`/home/dank/.pi/agent/models.json` on Dankware)
registers this endpoint under the Dankclaw provider with the **exact** FLM
registry tag as the model `id` (unknown tags answer a 200-wrapped error, not
a silent 1B fallback), `baseUrl` `http://192.168.0.143:52626/v1`, and a per-request context
a silent 1B fallback), `baseUrl` `http://192.168.0.143:7771/v1`, and a per-request context
window equal to the serving ctx-len. Health checks for this service must use
`GET /v1/version` (there is no `/health` route).
serverAppConfigs-docs/LLM/index.md
+1 -1
View File
@@ -80,7 +80,7 @@ ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box (frank-default-large)
- [[frank-default-sightsound]] — ACTIVE FLM NPU sight+sound runbook on
Dankclaw: Gemma4 E4B vision+audio pack on the XDNA2 NPU (FLM v1.0.1, port
52626, whole-registry serving, per-request model routing), incl. the
7771, whole-registry serving, per-request model routing), incl. the
IOMMU/SVA root cause and the `iommu=pt` fix behind its first-deploy crash
## CIRU Reference (Dankclaw)
serverAppConfigs-docs/index.md
+1 -1
View File
@@ -128,7 +128,7 @@ The LLM inference fleet on the two hosts. Full index:
- [[Dankclaw System Specs]] — the CIRU ROCmFPX box hardware
- [[Dankclaw Improvements]] — host-level changes on the CIRU box
- [[frank-default-sightsound]] — ACTIVE FLM NPU sight+sound runbook on Dankclaw:
Gemma4 E4B vision+audio pack on the XDNA2 NPU, port 52626; whole local
Gemma4 E4B vision+audio pack on the XDNA2 NPU, port 7771; whole local
registry served on one port with per-request model routing; includes the
IOMMU/SVA root cause and the `iommu=pt` fix behind its first-deploy crash