docs: frank-default-sightsound runbook — model swap + tag switching

serverAppConfigs-docs/LLM/frank-default-sightsound.md

@@ -1,7 +1,7 @@
 ---
 type: configuration
 title: frank-default-sightsound Runbook
-description: FastFlowLM (FLM) NPU sight+sound endpoint on Dankclaw — XDNA2 NPU serving the Gemma4 E4B vision+audio pack on port 7771, including the IOMMU/SVA root cause that took it down at first deploy.
+description: FastFlowLM (FLM) NPU sight+sound endpoint on Dankclaw — XDNA2 NPU serving Qwen3.8 Distilled Heretic vision+text pack on port 7771, including the IOMMU/SVA root cause that took it down at first deploy.
 tags:
   - flm
   - fastflowlm
@@ -9,15 +9,15 @@ tags:
   - npu
   - dankclaw
   - frank-default-sightsound
-timestamp: 2026-08-30
+timestamp: 2026-09-01
 ---
 
 # frank-default-sightsound — FLM NPU sight+sound Runbook
 
 `frank-default-sightsound` is the third inference endpoint on Dankclaw: a
 FastFlowLM (FLM) v1.0.1 server (`/usr/bin/flm`, dpkg package `fastflowlm`)
-running on the **XDNA2 NPU** (50 TOPS). It exists to serve *sight and sound* —
-image and audio input — which the iGPU llama.cpp fleet ([[kingjones-strix-rocmfp4]],
+running on the **XDNA2 NPU** (50 TOPS). It serves *sight* (vision) —
+which the iGPU llama.cpp fleet ([[kingjones-strix-rocmfp4]],
 `frank-default-large`) does not do. The model-selection tradeoff: the only
 NPU2 pack family on this box with BOTH a vision and an audio tower is the
 Gemma4 E4B IT pack; the uncensored candidate (`qwythos:9b`, 8.94 GB, 262k ctx)
@@ -34,14 +34,44 @@ is text-only and fails the hard modality requirement, so this endpoint trades
 | Deploy | `scripts/deploy.sh dankclaw --service frank-default-sightsound` (relink + daemon-reload + enable + restart; health gate polls `/v1/version`, 300 s cap — FLM has **no `/health` route**) |
 | Port | `7771` on `0.0.0.0` (52625/52626 left free for ad-hoc `flm serve` experiments) |
 | Power mode | `FLM_PMODE=turbo` — no other tenant uses the NPU on this host (the large model runs on the iGPU), so the top power mode costs no neighbour anything; drop to `performance` if iGPU power-budget contention ever shows |
-| Context | `FLM_CTX_LEN` = verified max for the E4B pack on this box (native 128K context; the registry's default context length is only the fallback). FLM rounds non-power-of-2 values up. |
-| ASR | `FLM_ASR=0` — the E4B pack answers audio natively through its own audio tower in `/v1/chat/completions`; setting `FLM_ASR=1` additionally loads Whisper-V3-Turbo-NPU2 (+0.62 GB) and exposes `/v1/audio/transcriptions` |
+| Context | `FLM_CTX_LEN=131072 (128K; verified max on this box; native context length.) |
+| ASR | `FLM_ASR=0` — the E4B pack answers images natively through its vision tower in `/v1/chat/completions`; setting `FLM_ASR=1` additionally loads Whisper-V3-Turbo-NPU2 (+0.62 GB) and exposes `/v1/audio/transcriptions` |
 
 The serve script exports `FLM_CONFIG_PATH=/home/dank/.config/flm/model_list.json`
 and `FLM_XCLBIN_PATH=/home/dank/.config/flm/xclbins`; kernels are symlinked
 from `/opt/fastflowlm/share/flm/xclbins/` into the per-pack directories. A tag
 swap is therefore an env-file edit, not a script edit.
 
+## Model tags for switching
+
+FLM routes per-request by the exact registry tag in the `model` field. To swap
+the service model, change `FLM_MODEL` in the env file and restart. To let
+clients choose, send requests with the desired `model` field.
+
+| Tag | Model | Modality | Size | Family |
+|-----|-------|----------|------|--------|
+| `qwen3.8-distilled-heretic:9b` | Qwen3.8 Distilled Heretic | vision + reasoning | 7.6 GB | qwen3.5 |
+| `gemma4-it:e4b` | Gemma4 E4B IT | vision + audio | 9.1 GB | gemma4e |
+| `qwythos:9b` | Qwythos 9B Claude Mythos | vision + reasoning | 8.9 GB | qwen3.5 |
+| `gpt-oss:20b` | GPT-OSS 20B | text only | 13.8 GB | gpt-oss |
+
+**To swap the active model** (changes what the service loads at start):
+```
+sed -i 's/^FLM_MODEL=.*/FLM_MODEL=<tag>/' scripts/frank-default-sightsound.env
+./scripts/deploy.sh dankclaw --service frank-default-sightsound
+```
+
+**To switch per-request** (service stays loaded, client sends the tag):
+```
+curl http://192.168.0.143:7771/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{"model":"gemma4-it:e4b","messages":[...],"max_tokens":64}'
+```
+
+Known tags are in the user registry at
+`/home/dank/.config/flm/model_list.json`; unknown tags answer HTTP 200 with
+`{"error":"Failed to load <tag> model!"}` (no choices).
+
 ## Serving model and API contract
 
 - The endpoint serves the **whole local registry** (36 tags at first deploy) on